\documentclass[twoside,letterpaper,11pt]{article}

%% LaTeX - Article customise

%%% PACKAGES
%\usepackage{booktabs} % for much better looking tables
%\usepackage{array} % for better arrays (eg matrices) in maths
%\usepackage{paralist} % very flexible & customisable lists (eg. enumerate/itemize, etc.)
\usepackage{verbatim} % adds environment for commenting out blocks of text & for better verbatim
%\usepackage{sbfigure} % make it possible to include more than one captioned figure/table in a single float
% These packages are all incorporated in the memoir class to one degree or another...

%%% PAGE DIMENSIONS
\usepackage{geometry} % to change the page dimensions
\geometry{margin=0.8in} % for example, change the margins to 2 inches all round
%\geometry{landscape} % set up the page for landscape
% read geometry.pdf for detailed page layout information

%%% HEADERS & FOOTERS
\usepackage{fancyhdr} % This should be set AFTER setting up the page geometry
\pagestyle{fancy} % options: empty , plain , fancy
\renewcommand{\headrulewidth}{0pt} % customise the layout...
\lhead{}\chead{}\rhead{}
\lfoot{}\cfoot{\thepage}\rfoot{}

%%% SECTION TITLE APPEARANCE
%\usepackage{sectsty}
%\allsectionsfont{\sffamily\mdseries\upshape} % (See the fntguide.pdf for font help)
% (This matches ConTeXt defaults)

%%% ToC APPEARANCE
\usepackage[nottoc,notlof,notlot]{tocbibind} % Put the bibliography in the ToC
\usepackage[titles]{tocloft} % Alter the style of the Table of Contents
\renewcommand{\cftsecfont}{\rmfamily\mdseries\upshape}
\renewcommand{\cftsecpagefont}{\rmfamily\mdseries\upshape} % No bold!

%% END Article customise

\usepackage{amsmath}
\usepackage{times}
%\renewcommand{\tt}[1]{{\texttt {#1}}}
%\renewcommand{\tt}[1]{\rm #1}
\newcommand{\tmvversion}{0.62}
\newcommand{\myemailx}{mike\_jarvis@users.sourceforge.net}
\newcommand{\myemail}{\href{mailto:\myemailx}{\myemailx}}
%\newcommand{\website}{http://www.hep.upenn.edu/$\sim$mjarvis/tmv/}
\newcommand{\websitex}{http://sourceforge.net/projects/tmv-cpp/}
\newcommand{\website}{\url{\websitex}}

\usepackage[bookmarks,bookmarksnumbered]{hyperref}
\usepackage{listings}
\usepackage[usenames,dvips]{color}
\usepackage{makeidx}
\makeindex

\lstset{
language=C++,
basicstyle=\ttfamily, 
morecomment=[l][\color{blue}]{//},
morecomment=[l][\color{red}]{//!},
morekeywords={size_t},
showstringspaces=false,
keywordstyle=\color{Green},
%stringstyle=\color{Maroon},
%identifierstyle=\color{},
%classoffset=1,
%morekeywords={RAND_MAX},
%keywordstyle=\color{BrickRed},
classoffset=2,
morekeywords={Vector,ListInit,Error,SumAbsElements,Norm2,Norm,NormInf,SumElements,
MinAbsElement,MaxAbsElement,MinElement,MaxElement,AddToAll,ReverseSelf,Zero,
SetAllTo,MakeBasis,SubVector,Reverse,FortranStyle,Real,Imag,Conjugate,Sort,
ReversePermute,Permute,DESCEND,REAL_COMP,ASCEND,IMAG_COMP,
ABS_COMP,ARG_COMP,tmv},
keywordstyle=\color{black}
}

%\renewcommand{\tt}[1]{{\texttt {#1}}}
%\newenvironment{tmvcode}{\endgraf\noindent \endgraf\verbatim}{\endverbatim} 
%\newcommand{\caret}{\mbox{$^{}$}}
%\newcommand{\inputcode}[1]{\verbatiminput{#1}}

\renewcommand{\tt}[1]{{\lstinline {#1}}}
\lstnewenvironment{tmvcode}{}{}
%\newcommand{\caret}^
\newcommand{\inputcode}[1]{\lstinputlisting[basicstyle=\small]{#1}}

\begin{document}

\title{Template Matrix/Vector Library for C++ \\ User Manual \\ Version \tmvversion}
\author{Mike Jarvis}
\date{January 20, 2009}
\maketitle

\tableofcontents

\newpage
\section{Overview}

First, this library is provided without any warranty of any kind.  There is no guarantee
that the code will produce accurate results for all inputs.  If someone dies because
my code gave you a wrong result, \underline{do not} blame me.

OK, that was mostly for the crazies out there.  Really, I think this is a pretty good 
product, and I've been using it for my own research extensively.  So at least 
for the routines that I use, they are probably reasonably well debugged.
I also have a test suite, which tries to be comprehensive, although
occasionally I find bugs that I hadn't thought to test for in the test suite, so 
there may still be a few lurking in there.  That means the code should definitely be considered
a beta-type release.  Hence the ``0.'' version number.  I'm also still adding functionality,
so there may be interface changes from one version to the next if I decide I 
want to do something a different way.  Just be warned.  Any such changes will be 
mentioned in \S\ref{history} of subsequent releases.

The Template Matrix/Vector (TMV) Library is a C++ class library designed to make
writing code with vectors and matrices both transparent and fast.  Transparency 
means that when you look at your code months later, it is obvious what the code
does, making it easier to debug.  Fast means the execution time of the code should
be as fast as possible - this is mostly algorithm dependent, so we want the 
underlying library code to use the fastest algorithms possible.

If there were another free C++ Matrix library available that satisfied these requirements
and provided all (or even most) of the functionality I wanted, I probably would
not have written this.  But, at least when I started writing this, the available matrix libraries
were not very good.  Either they didn't have good operator overloading, or they 
didn't do complex matrices well, or they didn't include singular value decompositions,
or something.  Anyway, since I did decide to write 
my own library, hopefully other people can benefit from my efforts and will find 
this to be a useful product.

Given the above basic guidelines, the specific design features that I 
have incorporated into the code include:

\begin{enumerate}
\item
\textbf{Operator overloading}

Matrix equations look like real math equations in the code. 
For example, one can write 
\begin{tmvcode}
v2 = m * v1;
v2 += m.Transpose() * v1;
m *= 3.;
v2 -= m1*v1 + 3*v2 + m2.Transpose()*v3;
\end{tmvcode}
to perform the corresponding mathematical operations.

I also define division:
\begin{tmvcode}
x = b/A;
\end{tmvcode}
to mean solve the matrix equation $A x = b$.  If $A$ has more rows than columns,
then the solution will be a least-square solution.

\item
\textbf{Delayed evaluation}

Equations like the above 
\begin{tmvcode}
v2 = m * v1;
v2 += m.Transpose() * v1;
\end{tmvcode}
do not create a temporary vector before assigning or adding the result to \tt{v2}.  This makes 
the TMV code just as fast as code which uses a functional form, such as:
\begin{tmvcode}
dgemv('N',m,n,1.,mp,ldm,v1p,s1,0.,v2p,s2);
dgemv('T',m,n,1.,mp,ldm,v1p,s1,1.,v2p,s2);
\end{tmvcode}
In fact, when installed with a BLAS library, the first version with TMV just calls those exact 
BLAS functions given by the second version.  
So speed is not sacrificed for the sake of code legibility.

However, a more complicated equation like
\begin{tmvcode}
v2 += m1*v1 + 3*v2 + m2.Transpose()*v3;
\end{tmvcode}
does not have a specialized routine, 
so it will require a couple temporary \tt{Vector}s.
Generally, if a statement performs just one operation, no temporary will be needed.  
(This includes all operations with corresponding BLAS functions along with some others
that are not included in BLAS.)
More complicated equations like this last example will give the right answer, 
but may not be quite as efficient as if you expand
the code to perform one operation per line.

\item
\textbf{Template classes}

Of course, all of the matrix and vector classes are templates, so you can have 
\begin{tmvcode}
Matrix<float>
Matrix<double>
Matrix<complex<double> >
Matrix<long double>
Matrix<MyQuadPrecisionType>
\end{tmvcode}
or whatever.

\item
\textbf{Mix complex/real}

One can multiply a real matrix by a complex
vector without having to copy the matrix to a complex one for the sake of the calculation
and deal with the concomitantly slower calculation.  
Likewise for the other arithmetic operations.  

However, it does not allow mixing of underlying data types 
(float with double, for example), 
with the exception of simple assignments.

\item
\textbf{Views}

Operations like \tt{m.Tranpose()} or \tt{v.SubVector(3,8)}
return ``views'' of the underlying data rather than copying to new storage.  
This model helps delay calculations, which increases efficiency.  And the syntax
is fairly obvious.  For example:
\begin{tmvcode}
v.SubVector(3,8) *= 3.;
m.row(3) += 4. * m.row(0);
m *= m.Transpose();
\end{tmvcode}
modifies the underlying \tt{v} and \tt{m} in the obvious ways.

Note that in the last equation, \tt{m.Transpose()} uses the same storage as
\tt{m}, which is getting overwritten in the process.  The code recognizes 
this conflict and uses temporary storage to obtain the correct result.
See ``Alias checking'' below for more about this.

\item
\textbf{C- or Fortran-style indexing}

Both C- and Fortran-style (i.e. zero-based or one-based) indexing are possible for
element access of the matrices and vectors.

With C-style indexing, all matrix and vector indexing starts with 0.  
So the upper-left element of a matrix is \tt{m(0,0)}, not \tt{m(1,1)}. 
Likewise, the lower right element of an $M \times N$ matrix is \tt{m(M-1,N-1)}.
For element ranges, such as \tt{v.SubVector(0,10)}, the first number is the 
index of the first element, and the second number is ``one-past-the-end'' of 
the range.  So, this would return a 10 element vector from \tt{v(0)} to
\tt{v(9)} inclusive, not an 11 element one including \tt{v(10)}.

With Fortran-style indexing, all matrix and vector indexing starts with 1.
So the upper-left element of a matrix is \tt{m(1,1)}.  Likewise, the lower right
element of an $M \times N$ matrix is \tt{m(M,N)}.  For element ranges, such as
\tt{v.SubVector(1,10)}, the first number is the index of the first element, and
the second number is the last element.  So, this would return a 10 element vector
from \tt{v(1)} to \tt{v(10)} inclusive, which represents the same actual elements 
as the C-style example above.

\item
\textbf{Special matrices}

Many applications use matrices with known sparsity or a specific structure.  
The code is able to exploit a number of these structures for increased
efficiency in both speed and storage.  So far the following special matrices are
available: diagonal, upper/lower triangle, symmetric, hermitian, banded, and 
symmetric or hermitian banded.  
Special types of banded matrices, such as 
upper and lower banded, tridiagonal, or Hessenberg, may all be declared as a 
regular BandMatrix.  The code checks the number of sub- and super-diagonals 
and uses the right algorithm when such specialization is advantageous for a 
particular calculation.

\item
\textbf{Flexible storage}

Both row-major and column-major storage are possible as an optional
extra template parameter.
For banded matrices, there is also diagonal-major storage.
This can aid I/O, which may require a particular format to mesh with another
program.  Also, some algorithms 
are faster for one storage than than the other, so it can be worth switching storage
and checking the speed difference.

\item
\textbf{Alias checking}

Expressions such as \tt{m *= m} pose a problem for many matrix libraries, since
no matter what order you do the calculation, you will necessarily overwrite elements 
that you need for later stages of the calculation.  
The TMV code automatically recognizes the
conflict (generally known as an alias) and creates the needed temporary storage.

The code only checks the addresses of the first elements of the different objects.
So expressions such as \tt{m = m.LowerTri() * m.UpperTri()} will work
correctly, even though there are three types of matrices involved, since the 
address of the upper-left corner of each matrix is the same.  (This particular
expression does not even need a temporary.  The code orders the steps of this
calculation so it can be done in place.)

However, \tt{v.SubVector(5,15) += v.SubVector(0,10)} will be calculated 
incorrectly, since the subvectors start at different locations, so the code
doesn't notice the aliasing.  Here, elements 5-9 will be overwritten before
they are added to the left-side vector.

Therefore, some care is still needed.  But this limited check is sufficient
for most applications.

\item
\textbf{BLAS}

For the combinations of types for which there are existing BLAS
routines, the code can call the optimized BLAS routines instead of its own 
code.  For other combinations (or for user defined types like 
\tt{MyQuadPrecisionType} or somesuch), 
the code does its best to order the steps of the calculation, including
using blocking techniques, to be 
reasonably efficient, but it is definitely not as fast as BLAS for
most machines.

This feature can be turned off at compile time if desired with the 
compilation flag -DNOBLAS, although this is not generally recommended if 
BLAS is available on your machine, and speed is important for your 
application.  For some operations, the BLAS routines can be a factor of 
10 or more faster than the native TMV code.  (I'm working on reducing this
gap, but I don't really expect to ever do better than a factor of 2 slower
than an optimized BLAS library.)

\item
\textbf{LAPACK}

When possible, the code can call LAPACK routines, which
may be faster than the native TMV code.  For types that don't have LAPACK routines, 
the code uses blocked and/or recursive algorithms, which are similarly fast.  
Again, this feature can be turned off at compile time, this time with -DNOLAP
(-DNOBLAS necessarily turns off the LAPACK calls as well.)

For almost all algorithms, 
the TMV code is approximately as fast as LAPACK routines -
sometimes faster, since most
LAPACK distributions do not use recursive algorithms yet, which are generally
slightly faster on modern machines with good compilers. 
So if you don't want to deal with getting LAPACK
up and running, it won't generally be too bad, speedwise, 
to turn off the LAPACK calls.

The only real exception to this statement
is the eigenvector calculation for a hermitian matrix.  I have not yet 
implemented the new RRR
(Relatively Robust Representation) algorithm by Dhillon.  So if the code is
spending a significant time calculating eigenvectors, it may be worth having it call the
LAPACK routines.

\item
\textbf{Comments}

The code is highly commented, especially for the more complicated algorithms.
I have this egotistical quirk that I need to understand all of the code I write.
So none of the algorithms are just taken verbatim from LAPACK or anything like that.
I force myself to understand the algorithm (usually from Golub and van Loan,
but sometimes from a journal article or something I find on the net),
write a long comment explaining how it works, and then finally write the code
from scratch.  Then I usually compare my code to the LAPACK version, at which point
I have occasionally found ways to either speed it up or make it more accurate
(which also get commented, of course).

But the point is - if you want to understand how, say, the blocked Bunch-Kaufman
algorithm works, you'll probably do better looking at the comment in the 
TMV code than trying to decipher the LAPACK routines.  Plus, the code itself is
usually a lot more readable, since the arithmetic is written with the operator
overloads and such rather than the cryptic BLAS function names.

\end{enumerate}

All of the basic TMV classes and functions, including the \tt{Vector} and \tt{Matrix} 
classes, can be accessed with
\begin{tmvcode}
#include "TMV.h"
\end{tmvcode}
\index{TMV.h}
This file includes all the other files for the basic TMV routines.  The 
special matrices described below (other than diagonal and triangular matrices) 
are not included in this header file.
See their sections for the names
of the files to include to access those classes.

All of the TMV classes and functions reside in the namespace \tt{tmv}. 
And of course, they are all templates.
So if you want to declare a $10 \times 10$ \tt{Matrix}, one would write:
\begin{tmvcode}
tmv::Matrix<double> m(10,10);
\end{tmvcode}
\index{tmv namespace}

If writing \tt{tmv::} all the time is cumbersome, one can use \tt{using}
statements near the top of the code:
\begin{tmvcode}
using tmv::Matrix;
using tmv::Vector;
\end{tmvcode}
Or, while generally considered bad coding style, one can import the whole namespace:
\begin{tmvcode}
using namespace tmv;
\end{tmvcode}
Or, you could use typedef to avoid having to write the template type as well:
\begin{tmvcode}
typedef tmv::Matrix<double> DMatrix;
typedef tmv::Vector<double> DVector;
\end{tmvcode}
In this documentation, I will usually write \tt{tmv::} with the class names to help remind the reader
that it is necessary, especially near the 
beginnings of the sections.  
But for the sake of brevity and readability, I sometimes omit it.

Also, throughout most of the documentation, I will write \tt{T} for the underlying type.
Wherever you see \tt{T}, you should put \tt{double} or \tt{float} or whatever.
For a user defined type, like \tt{MyQuadPrecisionType}, for example, the main requirements are that 
in addition to the usual arithmetic operators, the functions:
\begin{tmvcode}
std::numeric_limits<T>::epsilon()
sqrt(T x) 
exp(T x) 
log(T x)
\end{tmvcode}
\index{User-defined types}
need to be defined appropriately, where \tt{T} is your type name.  See \S\ref{install} for details about
compiling the library for types other than \tt{double} and \tt{float}.

Some functions in this documentation will return a real value or require a real argument, 
even if \tt{T} is complex.  In these cases, I will write \tt{RT} to indicate 
``the real type associated with \tt{T}''.  Similarly, there are a couple places where \tt{CT}
indicates ``the complex type associated with \tt{T}''.

It may be worth noting that \tt{Matrix<int>} is possible as well if you compile the library
with the -DINST\_INT flag. 
However, only simple 
routines like multiplication and addition will give correct answers.  If you try to 
divide by a \tt{Matrix<int>}, for example, the required calculations are impossible 
for \tt{int}'s,
so the result will not be correct.  But since the possibility of multiplication of 
integer matrices seemed desirable, we do allow them to be used.  
\emph{Caveat programor}.  If debugging is turned on (or more accurately, not
turned off via the compile flag -DNDEBUG), then trying to do anything that requires
\tt{sqrt} or \tt{epsilon} for \tt{int}s will result in a runtime error.

There are three fonts used in this document.  First, times is used for the main text.  
Second, as you have no doubt already noticed, \tt{typewriter font} is used to 
indicate bits of code.  And finally, when I discuss the math about matrices, I 
use $italics$ - for example, $v_2 = m * v_1$.

Also, my syntax in this documentation is not very rigorous, aiming to maximize
readability of the code.  
I tend to freely mix the syntax with which a function is used with how
it is declared in order to try to provide all of the information you will need in one line.  
For example, the constructor listed as:
\begin{tmvcode}
tmv::Vector<T,index> v(size_t n, const T* vv)
\end{tmvcode}
is actually declared in the code as:
\begin{tmvcode}
tmv::Vector<T,index>::Vector(size_t n, const T* vv);
\end{tmvcode}
and when it is used in your source code, you would write something like:
\begin{tmvcode}
size_t n = 5;
const double vv[n] = {1.2, 3.1, 9.1, 9.2, -3.5};
tmv::Vector<double,tmv::CStyle> v(n,vv);
\end{tmvcode}
So, the notation that I use in the documentation for this constructor is kind of a hybrid between the declaration syntax and the use syntax.  The intent is to improve readability, but
if you are ever confused about how to use a particular method, you should look at
the \tt{.h} header files themselves, since they obviously have the exactly accurate
declarations.  

\newpage
\section{Vectors}
\index{Vector}

The \tt{Vector} class is our mathematical vector.  Not to be confused with
the standard template library's \tt{vector} class.  
Our \tt{Vector} class name is capitalized, while the STL \tt{vector} is not.
If this is not enough of a difference for you, and you are using both extensively in your code,
we recommend keeping the full \tt{tmv::Vector} designation for ours and \tt{std::vector}
for theirs to distinguish them.  Or you might want to \tt{typedef tmv::Vector} to some other name.

\tt{Vector} inherits from its base class \tt{GenVector} (i.e. ``generic vector'').
Most operations that do not 
modify the data are actually defined in \tt{GenVector} rather than \tt{Vector}, although
some routines are overridden in \tt{Vector} for speed reasons.  

The other classes that inherit from \tt{GenVector} are \tt{VectorView}, 
\tt{ConstVectorView} (both described in more detail below - see \S\ref{VViews}), and
\tt{VectorComposite}, which is the base class for all arithmetic operations that
return a (logical) \tt{Vector}.  This means that any of these other objects can be used
any time a \tt{Vector} can be used in a non-assignable capacity.  For example,
\tt{Norm(v1+m*v2)} is completely valid, and will automatically create the necessary
temporary \tt{Vector} to store the result of the mathematical operation in the parentheses.

There is another template argument for a \tt{Vector} in addition to \tt{T} (which
represents the data type of the elements).  The second template argument
may be either \tt{tmv::CStyle} or \tt{tmv::FortranStyle}.  Or it may be omitted,
in which case \tt{CStyle} is assumed.  This argument governs how the element
access is performed.

With C-style indexing, the first element of a \tt{Vector} of length \tt{N} is 
\tt{v(0)} and the last is \tt{v(N-1)}.  Also, methods that take range arguments
use the common C convention of ``one-past-the-end'' for the last element;
so \tt{v.SubVector(0,3)} returns a 3-element vector, not 4.

With Fortran-style indexing, the first element of the vector is \tt{v(1)} and the 
last is \tt{v(N)}.  And ranges indicate the first and last elements, so the same
subvector as above would now be accessed using \tt{v.SubVector(1,3)} to return
the first three elements.

All views of a \tt{Vector} keep the same indexing style as the original unless you
explicitly change it with a cast.  You can cast a \tt{VectorView<T,CStyle>} as
a \tt{VectorView<T,FortranStyle>} and vice versa.  Likewise for
\tt{ConstVectorView}.  

The only thing to watch out for about the indexing is that \tt{GenVector}
and \tt{VectorComposite} do not have the extra \tt{index} 
template argument and are always
indexed using the C-style convention.  Therefore, 
if you want to index a \tt{GenVector}
using the Fortran-style convention, you would need to recast it
as an object of type
\tt{ConstVectorView<T,FortranStyle>}.  A \tt{VectorComposite} would 
not generally be indexed, but if you did want to do so using the 
Fortran-style conventions, you would need to explicitly instantiate it
as a \tt{Vector<T,FortranStyle>}.

\subsection{Constructors}
\index{Vector!constructors}

Here, \tt{T} is used to represent the data type of the elements of the \tt{Vector}
(e.g. \tt{double}, \tt{complex<double>}, \tt{int}, etc.) and \tt{index} is either
\tt{tmv::CStyle} or \tt{tmv::FortranStyle}.  In all of the constructors the
\tt{index} template argument may be omitted, in which case \tt{CStyle} is assumed.

\begin{itemize}
\item 
\begin{tmvcode}
tmv::Vector<T,index> v(size_t n)
\end{tmvcode}
Makes a \tt{Vector} of size \tt{n} with \underline{uninitialized} values.
If debugging is turned on (this is actually the default - 
turn off debugging by compiling
with -DNDEBUG), then the values are in fact initialized to 888.  
This should help you notice
when you have neglected to initialize the \tt{Vector} correctly.

\item
\begin{tmvcode}
tmv::Vector<T,index> v(size_t n, T x)
\end{tmvcode}
Makes a \tt{Vector} of size \tt{n} with all values equal to \tt{x}

\item
\begin{tmvcode}
tmv::Vector<T,index> v(size_t n, const T* vv)
tmv::Vector<T,index> v(const std::vector<T>& vv)
\end{tmvcode}
Make a \tt{Vector} which copies the elements of \tt{vv}.
For the first one, \tt{n} specifies the length.  The second gets
the length from \tt{vv}.

\item
\begin{tmvcode}
tmv::Vector<T,index> v = tmv::BasisVector<T,index>(size_t n, int i)
\end{tmvcode}
Makes a \tt{Vector} whose elements are all \tt{0}, except \tt{v(i) = 1}.
Note the \tt{BasisVector} also has the \tt{index} template argument to 
indicate which element is meant by \tt{v(i)}.  Again, if it is omitted,
\tt{CStyle} is assumed.

\item
\begin{tmvcode}
tmv::Vector<T,index> v1(const tmv::GenVector<T2>& v2)
v1 = v2
\end{tmvcode}
Copy the \tt{Vector v2}, which may be of any type \tt{T2} so long
as values of type \tt{T2} are convertible into type \tt{T}.
The assignment operator has the same flexibility.

\item
\begin{tmvcode}
tmv::VectorView<T,index> v = 
      tmv::VectorViewOf(T* vv, size_t n, int step = 1)
tmv::ConstVectorView<T,index> v = 
      tmv::VectorViewOf(const T* vv, size_t n, int step = 1)
\end{tmvcode}
\index{Vector!view of raw memory}
Makes a \tt{VectorView} (see \S\ref{VViews} below) which refers to the exact
elements of \tt{vv}, not copying them to new storage.  The parameter \tt{n}
is the number of values to include in the view.
The optional \tt{step} parameter allows a non-unit spacing between 
successive vector elements in memory. 

\end{itemize}

\subsection{Initialization}
\index{Vector!initialization}

There are a number of ways to initialize the elements of a \tt{Vector}:

The most straightforward way is to simply set each element individually:
\begin{tmvcode}
for(int i=0; i<N; ++i) {
    v[i] = /* whatever */
}
\end{tmvcode}

But this is often inconvenient, especially if you want to assign a particular list of 
values that does not lend itself to being put in a \tt{for} loop.  Of course you can assign
each element one line at a time, but that can be a bit unwieldy.  So another 
way is to use a C array initializer, and then create the \tt{Vector} from these values.  For example:
\begin{tmvcode}
double var[5] = { 1.2, 3.2, -9.2, -1, 3.3 };
tmv::Vector<double> v(5,var);
\end{tmvcode}
This works, but it seems a bit inefficient to use a temporary variable.  The ideal notation
in my opinion would be to put the list initializer as a parameter in the constructor:
\begin{tmvcode}
tmv::Vector<double> v(5, 
    { 1.2, 3.2, -9.2, -1, 3.3 }); // Error - invalid
\end{tmvcode}
Unfortunately, there is not valid C++ syntax\footnote{
Apparently they are working on this shortcoming as part of the new so-called C++0x 
standards update.}.
One way to get around this is to use:
\begin{tmvcode}
tmv::Vector<double> v(5, 
    (double[]) { 1.2, 3.2, -9.2, -1, 3.3 }); // Usually OK
\end{tmvcode}
although not all compilers accept this code as valid.  I'm not even sure whether
it is ANSI-compliant C++ code or not\footnote{
The common editor, vim, doesn't seem to think so.  
It flags the braces in red, which is its indicator for a syntax error.  But most compilers
seem to compile it correctly.}.

So to address this issue, we have included a way to initialize a \tt{Vector} through a little
bit of C++ magic.  You can initialize a \tt{Vector} with:
\begin{tmvcode}
tmv::Vector<double> v(5);
v = tmv::ListInit, 
    1.2, 3.2, -9.2, -1, 3.3;
\end{tmvcode}
\index{ListInit}
The \tt{tmv::ListInit} object is just a sentinel to mark the start of a comma-delimited list
of values.  There must be precisely as many values as there are elements in the \tt{Vector},
or a \tt{tmv::ReadError} will be thrown.
\index{Exceptions!ReadError}

One advantage of this method is that the values do not need to be simple numbers.
Normally the elements in a list initializer need to be numerical literals, or at least to be 
computable at compile time.
With the \tt{ListInit} method, the elements can be variables, 
or even values returned from functions.  Anything that is convertible
into the element type of the \tt{Vector} will work.

Also, \tt{v} can be re-initialized with this method at any time, so it is not limited to 
initialization at construction.  And finally, \tt{v} can be a \tt{VectorView}, not just a \tt{Vector}.

\subsection{Access}
\index{Vector!access methods}
\label{vectoraccess}

\begin{itemize}
\item
\begin{tmvcode}
size_t v.size() const
\end{tmvcode}
\index{Vector!methods!size}
Returns the size (length) of \tt{v}.

\item
\begin{tmvcode}
T v[int i] const
T v(int i) const
T v.cref(int i) const
typename Vector<T>::reference v[int i]
typename Vector<T>::reference v(int i)
typename Vector<T>::reference v.ref(int i)
\end{tmvcode}
\index{Vector!methods!operator()}
\index{Vector!methods!operator[]}
\index{Vector!methods!cref}
\index{Vector!methods!ref}
The \tt{[]} and \tt{()} forms are equivalent.  Each returns the \tt{i}-th element of \tt{v}.  
With \tt{index = CStyle}, the first 
element is \tt{v(0)}, and the last element is \tt{v(n-1)}.
With \tt{index = FortranStyle}, they are \tt{v(1)} and \tt{v(n)}.

If \tt{v} is a 
\tt{const Vector}, a \tt{ConstVectorView}, or a \tt{GenVector}, 
then the return type is just the value, not a reference.

If \tt{v} is a 
non-\tt{const Vector}, then the return type is a normal reference, \tt{T\&}.

If \tt{v} is a \tt{VectorView}, then the return type is an object, which is
an lvalue (i.e. it is assignable), but which may not be \tt{T\&}.
Specifically, it has the type \tt{typename VectorView<T>::reference}.
For a real-typed \tt{VectorView}, it is just \tt{T\&}.  But for a
complex-typed \tt{VectorView}, the return type is an object that keeps track of the
possibility of a conjugation.

The main difference between the operator forms and \tt{cref} or \tt{ref} is that the latter versions do not
check for the validity of the parameter \tt{i}, even when compiling with debugging turned on.
Also, \tt{cref} and \tt{ref} always use \tt{CStyle} indexing.


\item
\begin{tmvcode}
typename tmv::Vector<T>::iterator v.begin()
typename tmv::Vector<T>::iterator v.end()
typename tmv::Vector<T>::const_iterator v.begin() const
typename tmv::Vector<T>::const_iterator v.end() const
typename tmv::Vector<T>::reverse_iterator v.rbegin()
typename tmv::Vector<T>::reverse_iterator v.rend()
typename tmv::Vector<T>::const_reverse_iterator v.rbegin() const
typename tmv::Vector<T>::const_reverse_iterator v.rend() const
\end{tmvcode}
\index{Vector!iterators}
\index{Vector!methods!begin}
\index{Vector!methods!end}
\index{Vector!methods!rbegin}
\index{Vector!methods!rend}
These provide iterator-style access into a \tt{Vector}, which works just like
the standard template library's iterators.  If \tt{v} is a \tt{VectorView},
the iterator types are slightly different from the \tt{Vector} iterators, 
so you should declare them as \tt{typename tmv::VectorView<T>::iterator},
etc. instead.

\end{itemize}

\subsection{Views}
\index{Vector!VectorView}
\index{Vector!views}
\label{VViews}

A \tt{VectorView<T>} object refers to the elements of some other object, such as a regular
\tt{Vector<T>} or \tt{Matrix<T>},
so that altering the elements in the view alters the
corresponding elements in the original object.  A \tt{VectorView}
can have non-unit
steps between elements (for example, a view of a column of a row-major
matrix).  It can also be a conjugation of the original
elements, so that
\begin{tmvcode}
tmv::VectorView<double> cv = v.Conjugate();
cv(3) = z;
\end{tmvcode}
would actually set the original element, \tt{v(3)} to \tt{conj(z)}. 

Also, we have to 
keep track of whether we are allowed to alter the original values or
just look at them.  
Since we want to be able to pass these views around, it turns out that
the usual \tt{const}-ing doesn't work the way you would want.
Thus, there are two objects that are views of a
\tt{Vector}:
\tt{ConstVectorView} and \tt{VectorView}.  
The first is only allowed to view,
not modify, the original elements.  The second is allowed to modify them.
This distinction is akin to the \tt{const\_iterator} and \tt{iterator} types in the
standard template library.

One slightly non-intuitive thing about \tt{VectorView}s is that a 
\tt{const VectorView} is still mutable.  The \tt{const} in this case
means that one cannot change the components to which the view refers.
But a \tt{VectorView} is inherently an object that can be used to 
modify the underlying data, regardless of any \tt{const} in front of it.

The following methods return views to portions of a \tt{Vector}.
If \tt{v} is either a (non-\tt{const}) \tt{Vector}
or a \tt{VectorView}, then a \tt{VectorView} is returned.
If \tt{v} is a \tt{const Vector}, a \tt{ConstVectorView}, or any other \tt{GenVector},
then a \tt{ConstVectorView} is returned.  

\begin{itemize}
\item
\begin{tmvcode}
v.SubVector(int i1, int i2, int istep=1)
\end{tmvcode}
\index{Vector!methods!SubVector}
This returns a view to a subset of the original vector.
\tt{i1} is the first element in the subvector.
\tt{i2} is either ``one past the end'' (C-style) or the last element
(Fortran-style) of the subvector.
\tt{istep} is an optional step size.
Thus, if you have a \tt{Vector~v} of length 10, and you want to
multiply the first 3 elements by 2, with C-style indexing, you could write:
\begin{tmvcode}
v.SubVector(0,3) *= 2.;
\end{tmvcode}
To set all the even elements to 0, you could write:
\begin{tmvcode}
v.SubVector(0,10,2).Zero();
\end{tmvcode}
And then to output the last 4 elements of \tt{v}, you could write:
\begin{tmvcode}
std::cout << v.SubVector(6,10);
\end{tmvcode}

For Fortran-style indexing, the same steps would be accomplished by:
\begin{tmvcode}
v.SubVector(1,3) *= 2.;
v.SubVector(1,9,2).Zero();
std::cout << v.SubVector(7,10);
\end{tmvcode}

\item
\begin{tmvcode}
v.Reverse()
\end{tmvcode}
\index{Vector!methods!Reverse}
This returns a view whose elements are the same as \tt{v},
but in the reverse order

\item
\begin{tmvcode}
v.Conjugate()
\end{tmvcode}
\index{Vector!methods!Conjugate}
This returns the conjugate of a \tt{Vector} as a view, so it still points
to the same physical elements, but modifying this will set the 
actual elements in memory to the conjugate of what you set.  Likewise,
accessing an element will return the conjugate of the value in memory.

\item
\begin{tmvcode}
v.View()
\end{tmvcode}
\index{Vector!methods!View}
Returns a view of a \tt{Vector}.  This seems at first like a silly function to have, but
if you write a function that takes a mutable \tt{Vector} argument, and you want to
be able to pass it views in addition to regular \tt{Vector}s, it is easier to write the 
function once with a \tt{VectorView} parameter.  Then you only need a second 
function with a \tt{Vector} parameter, which calls the first function
using \tt{v.View()} as the argument:
\begin{tmvcode}
double foo(const tmv::VectorView<double>& v)
{ ... [modifies v] ... }
double foo(tmv::Vector<double>& v)
{ return foo(v.View()); }
\end{tmvcode}

If you are not going to be modifying \tt{v} in the function, you only need to
write one function, and you should use the base class \tt{GenVector} 
for the argument type:
\begin{tmvcode}
double foo(const tmv::GenVector<double>& v)
{ ... [doesn't modify v] ... }
\end{tmvcode}
The arguments could then be a \tt{const Vector}, a \tt{ConstVectorView},
or even a \tt{VectorComposite}.

\item
\begin{tmvcode}
v.Real()
v.Imag()
\end{tmvcode}
\index{Vector!methods!Real}
\index{Vector!methods!Imag}
These return views to the real and imaginary parts of a complex \tt{Vector}.
Note the return type is a real view in each case:
\begin{tmvcode}
tmv::Vector<std::complex<double> > v(10,std::complex<double>(1,4));
tmv::VectorView<double> vr = v.Real();
tmv::VectorView<double> vi = v.Imag();
\end{tmvcode}

\end{itemize}

\subsection{Functions of a vector }
\index{Vector!functions of}
\label{vectorfunctions}

Functions that do not modify the \tt{Vector} are defined in 
\tt{GenVector}, and so can be used for any type derived from \tt{GenVector}:
\tt{Vector, ConstVectorView, VectorView,} or \tt{VectorComposite}.

Functions that modify the \tt{Vector} are only defined for 
\tt{Vector} and \tt{VectorView}.

\subsubsection{Non-modifying functions}

Each of the following functions can be written in two ways, either as a method or a function.
For example, the expressions:
\begin{tmvcode}
double normv = v.Norm();
double normv = Norm(v);
\end{tmvcode}
are equivalent.  In either case, \tt{v} can be any \tt{GenVector}.
Also, several of the functions below have multiple equivalent names.  For example,
\tt{Norm1} and \tt{SumAbsElements} are equivalent, so you can use whichever one is
clearer to you in your situation.
And just to remind you, \tt{RT} refers to the 
real type associated with \tt{T}.  So if \tt{T} is either \tt{double} or \tt{complex<double>},
\tt{RT} would be \tt{double}.

\begin{itemize}

\item
\begin{tmvcode}
RT v.Norm1() const
RT Norm1(v)
RT v.SumAbsElements() const
RT SumAbsElements(v)
\end{tmvcode}
\index{Vector!methods!Norm1}
\index{Vector!functions of!Norm1}
\index{Vector!methods!SumAbsElements}
\index{Vector!functions of!SumAbsElements}
The 1-norm of \tt{v}: $||v||_1 = \sum_i |v(i)|$.

\item
\begin{tmvcode}
RT v.Norm2() const
RT Norm2(v)
RT v.Norm() const
RT Norm(v)
\end{tmvcode}
\index{Vector!methods!Norm}
\index{Vector!functions of!Norm}
\index{Vector!methods!Norm2}
\index{Vector!functions of!Norm2}
The 2-norm of \tt{v}: $||v||_2 = (\sum_i |v(i)|^2)^{1/2}$.
This is the most common meaning for the norm of a vector, so we
define the \tt{Norm} function to be the same as \tt{Norm2}.

\item
\begin{tmvcode}
RT v.NormSq(const RT scale=1) const
RT NormSq(v)
\end{tmvcode}
\index{Vector!methods!NormSq}
\index{Vector!functions of!NormSq}
The square of the 2-norm of \tt{v}: $(||v||_2)^2 = \sum_i |v(i)|^2$.
In the method version of this function, you may provide an optional scale factor,
in which case the return value is equal to NormSq(scale*v) instead, 
which can help avoid underflow or overflow problems.

\item
\begin{tmvcode}
RT v.NormInf() const
RT NormInf(v)
RT v.MaxAbsElement() const
RT MaxAbsElement(v)
\end{tmvcode}
\index{Vector!methods!NormInf}
\index{Vector!functions of!NormInf}
\index{Vector!methods!MaxAbsElement}
\index{Vector!functions of!MaxAbsElement}
The infinity-norm of \tt{v}: $||v||_\infty = \max_i |v(i)|$.

\item
\begin{tmvcode}
RT v.MaxAbsElement(int* i=0) const
RT MaxAbsElement(v)
RT v.MinAbsElement(int* i=0) const
RT MinAbsElement(v)
T v.MaxElement(int* i=0) const
T MaxElement(v)
T v.MinElement(int* i=0) const
T MinElement(v)
\end{tmvcode}
\index{Vector!methods!MaxElement}
\index{Vector!functions of!MaxElement}
\index{Vector!methods!MinElement}
\index{Vector!functions of!MinElement}
\index{Vector!methods!MaxAbsElement}
\index{Vector!functions of!MaxAbsElement}
\index{Vector!methods!MinAbsElement}
\index{Vector!functions of!MinAbsElement}
The maximum/minimum element either by absolute value (the first four cases) or actual value
(the last four cases).  For complex values, there is no way to define a 
max or min element, so just the real component of each element is used.
The \tt{i} argument is available in the method versions of these function, and it is optional.
If it is present (and not 0), then \tt{*i} is set to the 
index of the max/min element returned.

\item
\begin{tmvcode}
T v.SumElements() const
T SumElements(v)
\end{tmvcode}
\index{Vector!methods!SumElements}
\index{Vector!functions of!SumElements}
The sum of the elements of \tt{v} $= \sum_i v(i)$.

\end{itemize}


\subsubsection{Modifying functions}

The following functions are methods of both \tt{Vector} and \tt{VectorView},
and they work the same way in the two cases, although there may be speed differences
between them.  
All of these are usually written on a line by themselves.  However, they do
return the (modified) \tt{Vector}, so you can string them together if you want.
For example:
\begin{tmvcode}
v.Clip(1.e-10).ConjugateSelf().ReverseSelf();
\end{tmvcode}
\index{Vector!functions of!Norm}
would first clip the elements at \tt{1.e-10}, then conjugate each element, then finally reverse the
order of the elements.  (This would probably not be considered 
very good programming style, however.)
Likewise, the expression:
\begin{tmvcode}
foo(v.Clip(1.e-10));
\end{tmvcode}
which would first clip the elements at \tt{1.e-10},
then pass the resulting \tt{Vector} to the function \tt{foo}.  

\begin{itemize}

\item
\begin{tmvcode}
v.Zero();
\end{tmvcode}
\index{Vector!methods!Zero}
Clear the \tt{Vector v}.  i.e. Set each element to 0.

\item
\begin{tmvcode}
v.SetAllTo(T x);
\end{tmvcode}
\index{Vector!methods!SetAllTo}
Set each element to the value \tt{x}.

\item
\begin{tmvcode}
v.Clip(RT thresh)
\end{tmvcode}
\index{Vector!methods!Clip}
Set each element whose absolute value is less than \tt{thresh} equal to 0.
Note that \tt{thresh} should be a real value even for complex valued
\tt{Vector}s.

\item
\begin{tmvcode}
v.AddToAll(T x)
\end{tmvcode}
\index{Vector!methods!AddToAll}
Add the value \tt{x} to each element.

\item 
\begin{tmvcode}
v.ConjugateSelf()
\end{tmvcode}
\index{Vector!methods!ConjugateSelf}
Change each element into its complex conjugate.  
Note the difference between this and \tt{v.Conjugate()}, 
which returns a \underline{view} to a conjugated version of \tt{v} without
actually changing the underlying data.  This function, \tt{v.ConjugateSelf()},
does change the underlying data.

\item
\begin{tmvcode}
v.ReverseSelf()
\end{tmvcode}
\index{Vector!methods!ReverseSelf}
Reverse the order of the elements.  Note the difference between this and 
\tt{v.Reverse()} which returns a \underline{view} to the elements in 
reversed order.

\item
\begin{tmvcode}
v.MakeBasis(int i)
\end{tmvcode}
\index{Vector!methods!MakeBasis}
Set all elements to 0, except for \tt{v(i)} = 1.

\item
\begin{tmvcode}
v.Swap(int i1, int i2)
\end{tmvcode}
\index{Vector!methods!Swap}
Swap elements \tt{v(i1)} and \tt{v(i2)}.

\item
\begin{tmvcode}
v.Permute(const int* p)
v.Permute(const int* p, int i1, int i2)
v.ReversePermute(const int* p)
v.ReversePermute(const int* p, int i1, int i2)
\end{tmvcode}
\index{Vector!methods!Permute}
\index{Vector!methods!ReversePermute}
The first one performs a series of swaps: (\tt{v(0)},\tt{v(p[0])}), (\tt{v(1)},\tt{v(p[1])}), ... 
The second starts at \tt{i1} and ends at \tt{i2-1} rather than
doing the whole range from 0 to \tt{n-1}.
The last two work the same way, but do the swaps in the opposite order.

Note: The indices listed in a permutation array (\tt{p}) always use 
the C-style convention, even if \tt{v} uses Fortran-style indexing.

\item
\begin{tmvcode}
v.Sort(int* p=0, tmv::ADType ad=tmv::ASCEND, 
      tmv::COMPType comp=tmv::REAL_COMP)
\end{tmvcode}
\index{Vector!methods!Sort}
Sorts the \tt{Vector~v}, returning the swaps required in the array \tt{p}.
If you do not care about the swaps, you can set \tt{p = 0}.
Note: the returned permutation array, \tt{p}, uses the C-style indexing convention
even if \tt{v} uses Fortran-style indexing.

The second parameter, \tt{ad}, determines whether the sorted \tt{Vector} 
will have its elements in ascending or descending order.  The possible values
are \tt{ASCEND} and \tt{DESCEND}. 

The third parameter, \tt{comp}, determines what component of the
elements to use for the sorting.  This is especially relevant if T is complex, 
since complex values are not intrinsically sortable.
The possible values are
\tt{REAL\_COMP}, \tt{ABS\_COMP},
\tt{IMAG\_COMP}, and \tt{ARG\_COMP}.
Only the first two make sense for non-complex \tt{Vector}s.  

\item
\begin{tmvcode}
Swap(v1,v2)
\end{tmvcode}
\index{Vector!functions of!Swap}
Swap the corresponding elements of \tt{v1} and \tt{v2}.  Note that this does physically
swap the data elements, not just some pointers to the data.  That's because this function
is mostly called on views into larger data fields: for example, swapping two rows of a 
\tt{Matrix}.  In any case, it always takes $O(N)$ time, not $O(1)$.

\end{itemize}

\subsection{Arithmetic}
\index{Vector!arithmetic}

\subsubsection{Operators}

All the usual operators work the way you would expect for \tt{Vector}s.  For shorthand in the 
following list, I use \tt{x} for a scalar of type \tt{T} or \tt{RT}, and
\tt{v} for a \tt{Vector}.  When there are two \tt{Vector}s
listed, they may either be both of the same type \tt{T}, or one may be of type 
\tt{T} and the other of \tt{complex<T>}.  Whenever \tt{v} is an lvalue,
if may be either a \tt{Vector} or a \tt{VectorView}.  Otherwise, it may be any \tt{GenVector}.

Also, I use the notation \tt{[+-]} to mean either \tt{+} or \tt{-}, since
the syntax is generally the same for these operators.
Likewise, I use \tt{[*/]} when their syntax is equivalent.

\begin{tmvcode}
v2 = -v1;
v2 = x * v1;
v2 = v1 [*/] x;
v3 = v1 [+-] v2;
v [*/]= x;
v2 [+-]= v1;
x = v1 * v2;
\end{tmvcode}
The last one, \tt{v1 * v2},
returns the inner product of two vectors, which is a scalar.  That is, the product
is a row vector times a column vector.  
\index{Vector!arithmetic!inner product}

This is the only case (so far)
where the specific row or column orientation of a vector matters.  For the others
listed here, the left side and the right side are implied to be of the same orientation, but
that orientation is otherwise arbitrary.
Later, when we get to a matrix times a vector, the orientation 
of the vector will be inferred from context.

\subsubsection{Subroutines}

Each of the above equations use deferred calculation so that the sum or product is not calculated
until the storage is known.  The equations can even be a bit more complicated without 
requiring a temporary.  Here are some equations that do not require a temporary 
\tt{Vector} for the calculation:

\begin{tmvcode}
v2 = -(x1*v1 + x2*v2);
v2 += x1*(x2*(-v1));
v2 -= x1*(v1/=x2);
\end{tmvcode}

The limit to how complicated the right hand side can be without using a 
temporary is set by the 
functions that the code eventually calls to perform the calculation.  
While you shouldn't ever
need to use these directly, it may help you understand when the code will require 
temporary \tt{Vector}s.  If you do use these, note that the \tt{v} parameters
are \tt{VectorView}s, rather than \tt{Vector}s.  So you would need to 
call them with \tt{v.View()} if \tt{v} is a \tt{Vector}.

\begin{itemize}

\item
\begin{tmvcode}
MultXV(T x, const VectorView<T>& v)
\end{tmvcode}
\index{Vector!arithmetic!MultXV function}
Performs the calculation \tt{v *= x}.

\item
\begin{tmvcode}
MultXV(T x, const GenVector<T1>& v1, const VectorView<T>& v2)
\end{tmvcode}
\index{Vector!arithmetic!MultXV function}
Performs the calculation \tt{v2 = x*v1}.

\item
\begin{tmvcode}
AddVV(T x, const GenVector<T1>& v1, const VectorView<T>& v2)
\end{tmvcode}
\index{Vector!arithmetic!AddVV function}
Performs the calculation \tt{v2 += x*v1}.

\item
\begin{tmvcode}
AddVV(T x1, const GenVector<T1>& v1, T x2, const GenVector<T2>& v2,
      const VectorView<T>& v3)
\end{tmvcode}
\index{Vector!arithmetic!AddVV function}
Performs the calculation \tt{v3 = x1*v1 + x2*v2}.

\item
\begin{tmvcode}
T MultVV(const GenVector<T>& v1, const GenVector<T2>& v2)
\end{tmvcode}
\index{Vector!arithmetic!MultVV function}
Performs the calculation \tt{v1*v2}.

\end{itemize}

More complicated arithmetic equations such as
\begin{tmvcode}
v1 += x*(v1+v2+v3) + (x*v3-v1) 
\end{tmvcode}
will require one or more temporary vectors, and so may be less efficient than 
you might like, but the code should return the correct result, no matter
how complicated the equation is.

\subsection{Input/Output}
\index{Vector!input/output}

The simplest output is the usual:
\begin{tmvcode}
os << v
\end{tmvcode}
where \tt{os} is any \tt{std::ostream}.
The output format is:
\begin{tmvcode}
n ( v(0)  v(1)  v(2)  ...  v(3) )
\end{tmvcode}
where \tt{n} is the length of the \tt{Vector}.

The same format can be read back in one of two ways:
\begin{tmvcode}
tmv::Vector<T> v(n);
is >> v;
std::auto_ptr<tmv::Vector<T> > pv;
is >> pv;
\end{tmvcode}
For the first version, the \tt{Vector} must already be declared, which 
requires knowing how big it needs to be.  If the input \tt{Vector} does not
match in size, an exception of type \tt{tmv::ReadError} is thrown.
The second version allows you to automatically get the size from the input.  
The \tt{Vector} pointed to by \tt{pv}
will be created according to whatever size the input \tt{Vector} is.
\index{Exceptions!ReadError}

Often, it is convenient to output only those values that aren't very small. 
This can be done using
\begin{tmvcode}
v.Write(std::ostream& os, RT thresh)
\end{tmvcode}
\index{Vector!methods!Write}
which is equivalent to
\begin{tmvcode}
os << tmv::Vector<T>(v).Clip(thresh);
\end{tmvcode}
but without requiring the temporary \tt{Vector}.

\newpage
\section{Dense Rectangular Matrices}
\index{Matrix}

The \tt{Matrix} class is our dense matrix class.  It inherits from \tt{GenMatrix},
which has the definitions of all the methods that 
do not modify the \tt{Matrix}.

The other classes that inherit from \tt{GenMatrix} are \tt{ConstMatrixView},
\tt{MatrixView} (see \S\ref{MViews} below), and \tt{MatrixComposite}, which 
is the base class for the various arithmetic operations that return a 
(logical) \tt{Matrix}.  
\index{Matrix!MatrixView}
\index{Matrix!ConstMatrixView}
\index{Matrix!GenMatrix}
\index{Matrix!MatrixComposite}

\tt{GenMatrix} in turn inherits from \tt{BaseMatrix}.  All of the various
special \tt{Matrix} classes also inherit from \tt{BaseMatrix}.
\tt{BaseMatrix} has virtual declarations for the functions that can be 
performed on any kind of \tt{Matrix} regardless of its structure or sparsity.
\index{BaseMatrix}

In addition to the data type template parameter (indicated here by \tt{T} as usual),
there is also a storage template parameter, which may be either 
\tt{RowMajor} or \tt{ColMajor}.

Finally, there is also a template argument indicating which indexing convention
you want the matrix to use, which may be either \tt{CStyle} or 
\tt{FortranStyle}.

With C-style indexing, the upper-left element of an $M \times N$ \tt{Matrix} is
\tt{m(0,0)}, the lower-left is \tt{m(M-1,0)}, the upper-right is \tt{m(0,N-1)},
and the lower-right is \tt{m(M-1,N-1)}.  Also, methods that take a pair of 
indices to define a range use the common C convention of ``one-past-the-end'' 
for the meaning of the second index.
So \tt{m.SubMatrix(0,3,3,6)} returns a $3 \times 3$ submatrix.

With Fortran-style indexing, the upper-left element of an $M \times N$ \tt{Matrix}
is \tt{m(1,1)}, the lower-left is \tt{m(M,1)}, the upper-right is \tt{m(1,N)},
and the lower-right is \tt{m(M,N)}.  Also, methods that take range arguments
take the pair of indices to be the actual first and last elements in the range.
So \tt{m.SubMatrix(1,3,4,6)} returns the same $3 \times 3$ submatrix as given above.

All views of a \tt{Matrix} keep the same indexing style as the original unless you
explicitly change it with a cast.  You can cast a \tt{MatrixView<T,CStyle>}
as a \tt{MatrixView<T,FortranStyle>} and vice versa.  (Likewise for 
\tt{ConstMatrixView}.)  However, as for \tt{GenVector}, you should be
aware that \tt{GenMatrix}
and \tt{MatrixComposite} do not have the extra template argument and are always
indexed using the C-style convention.  So if you want to index one of these 
using the Fortran-style convention, you need to (respectively) cast the 
\tt{GenMatrix} as a \tt{ConstMatrixView<T,FortranStyle>} or instantiate the
\tt{MatrixComposite} as a \tt{Matrix<T,FortranStyle>}.

You may omit the indexing template argument, in which case \tt{CStyle} is assumed.
And if so, you may also then omit the storage argument, in which case \tt{ColMajor}
is assumed.  If you want to specify \tt{FortranStyle} indexing, you need
to include the storage argument.

\subsection{Constructors}
\index{Matrix!constructors}

We use \tt{stor} to indicate the storage template argument. 
This argument must be either 
\tt{tmv::RowMajor} or \tt{tmv::ColMajor}.  And \tt{index} indicates either 
\tt{tmv::CStyle} or \tt{tmv::FortranStyle}.  The default values for these
are \tt{tmv::ColMajor} and \tt{tmv::CStyle} if the arguments are omitted.

\begin{itemize}
\item 
\begin{tmvcode}
tmv::Matrix<T,stor,index> m(size_t nrows, size_t ncols)
\end{tmvcode}
Makes a \tt{Matrix} with \tt{nrows} rows and \tt{ncols} columns with 
\underline{uninitialized} values.
If debugging is turned on (this is actually the default -
turn off debugging by compiling
with -DNDEBUG), then the values are in fact initialized to 888.  
This should help you notice
when you have neglected to initialize the \tt{Matrix} correctly.

\item
\begin{tmvcode}
tmv::Matrix<T,stor,index> m(size_t nrows, size_t ncols, T x)
\end{tmvcode}
Makes a \tt{Matrix} with \tt{nrows} rows and \tt{ncols} columns with all 
values equal to \tt{x}

\item
\begin{tmvcode}
tmv::Matrix<T,stor,index> m(size_t nrows, size_t ncols, const T* vv)
tmv::Matrix<T,stor,index> m(size_t nrows, size_t ncols, 
      const std::vector<T>& vv)
\end{tmvcode}
Makes a \tt{Matrix} with \tt{nrows} rows and \tt{ncols} columns,
which copies the elements of \tt{vv}.

If \tt{stor} is \tt{RowMajor}, then the elements of \tt{vv} are taken to
be in row-major order: first the \tt{ncols} elements of the first row, then the
\tt{ncols} elements of the second row, and so on for the \tt{nrows} rows.
Likewise if \tt{stor} is \tt{ColMajor},
then the elements of \tt{vv} are taken to be in column-major order - 
the elements of the first column, then the second, and so on.

\item
\begin{tmvcode}
tmv::Matrix<T,stor,index> m(const std::vector<std::vector<T> >& vv)
\end{tmvcode}
Makes a \tt{Matrix} with elements \tt{m(i,j) = vv[i][j]}.  The size of
the \tt{Matrix} is taken from the sizes of the \tt{vector}s.  
(If \tt{index} is \tt{FortranStyle}, then \tt{m(i,j) = vv[i-1][j-1]}.)

\item
\begin{tmvcode}
tmv::Matrix<T,index> m1(const GenMatrix<T2>& m2)
m1 = m2
\end{tmvcode}
Copy the \tt{Matrix m2}, which may be of any type \tt{T2} so long
as values of type \tt{T2} are convertible into type \tt{T}.
The assignment operator has the same flexibility.

\item
\begin{tmvcode}
tmv::MatrixView<T,index> m = 
      tmv::MatrixViewOf(T* vv, size_t nrows, size_t ncols, 
      StorageType stor)
tmv::ConstMatrixView<T,index> m = 
      tmv::MatrixViewOf(const T* vv,  size_t nrows, size_t ncols, 
      StorageType stor)
\end{tmvcode}
\index{Matrix!view of raw memory}
Makes a \tt{MatrixView} (see \S\ref{MViews} below) that refers to the exact
elements of \tt{vv}, not copying them to new storage.

\item
\begin{tmvcode}
tmv::MatrixView<T,index> m = 
      RowVectorViewOf(VectorView<T,index> v)
tmv::ConstMatrixView<T,index> m = 
      RowVectorViewOf(ConstVectorView<T,index> v)
\end{tmvcode}
\index{Vector!view as a 1$\times$n matrix}
Makes a view of the \tt{Vector}, which treats it as a $1\times n$ \tt{Matrix}
(i.e. a single row).  

\item
\begin{tmvcode}
tmv::MatrixView<T,index> m = 
      ColVectorViewOf(VectorView<T,index> v)
tmv::ConstMatrixView<T,index> m = 
      ColVectorViewOf(ConstVectorView<T,index> v)
\end{tmvcode}
\index{Vector!view as an n$\times$1 matrix}
Makes a view of the \tt{Vector}, which treats it as an $n \times 1$ \tt{Matrix}
(i.e. a single column).  

\end{itemize}

\subsection{Initialization}
\index{Matrix!initialization}

A \tt{Matrix} can be initialized with a comma-delimited list using
the sentinel object, \tt{ListInit}, just like with a \tt{Vector}.  For example:
\begin{tmvcode}
tmv::Matrix<double,tmv::RowMajor> m(3,5);
m = tmv::ListInit, 
     1.2,  3.2, -9.2, -1.0,  3.3, 
     2.0, -2.7,  2.2, -6.5, -7.6, 
    -8.2,  3.2, -8.7,  5.1,  1.6;
\end{tmvcode}
\index{ListInit}
There must be precisely as many values as there are elements in the \tt{Matrix},
or a \tt{tmv::ReadError} will be thrown.
\index{Exceptions!ReadError}

This initialization is most readable when the \tt{Matrix} \tt{m} is \tt{RowMajor}, since then the 
elements in the list can be arranged as in the above example, which shows the
correct matrix structure of the elements.  If \tt{m} is \tt{ColMajor}, then the
list elements are assigned in order of the columns instead.

Also, the list initialization works for a \tt{MatrixView} as well.  However, it only works
if the \tt{MatrixView}'s elements are all contiguous in memory.  You can check
if this is the case with the \tt{CanLinearize()} method described in \S\ref{linearviews}.

\subsection{Access}
\index{Matrix!access methods}

\begin{itemize}
\item
\begin{tmvcode}
size_t m.nrows() const
size_t m.ncols() const
size_t m.colsize() const
size_t m.rowsize() const
\end{tmvcode}
\index{Matrix!methods!nrows}
\index{Matrix!methods!ncols}
\index{Matrix!methods!colsize}
\index{Matrix!methods!rowsize}
Returns the size of each dimension of \tt{m}.  \tt{nrows()} and \tt{colsize()} are equivalent.
Likewise \tt{ncols} and \tt{rowsize()} are equivalent.

\item
\begin{tmvcode}
T m(int i, int j) const
T m[int i][int j] const
T m.cref(int i, int j) const
typename tmv::Matrix<T>::reference m(int i, int j)
typename tmv::Matrix<T>::reference m[int i][int j]
typename tmv::Matrix<T>::reference m.ref(int i,int j)
\end{tmvcode}
\index{Matrix!methods!operator()}
\index{Matrix!methods!operator[]}
\index{Matrix!methods!ref}
\index{Matrix!methods!cref}
Returns the \tt{i,j} element of \tt{m}. i.e. the \tt{i}th element in the 
\tt{j}th column.  Or
equivalently, the \tt{j}th element in the \tt{i}th row. 

With C-style indexing, the upper-left element
of a \tt{Matrix} is \tt{m(0,0)}, the lower-left is \tt{m(nrows-1,0)},
The upper-right is \tt{m(0,ncols-1)}, and the lower-right is
\tt{m(nrows-1,ncols-1)}.

With Fortran-style indexing, these four elements would instead be
\tt{m(1,1)}, \tt{m(nrows,1)}, \tt{m(1,ncols)}, and \tt{m(nrows,ncols)},
respectively.

If \tt{m} is a 
\tt{const Matrix}, a \tt{ConstMatrixView}, or a \tt{GenMatrix}, 
then the return type is just the value, not a reference.

If \tt{m} is a 
non-\tt{const Matrix}, then the return type is a normal reference \tt{T\&}.

If \tt{m} is a \tt{MatrixView}, then the return type is an object,
which is an lvalue (i.e. it is assignable), but which may not be \tt{T\&}.
It has the type \tt{typename MatrixView<T>::reference}, 
(which is the same as \tt{typename VectorView<T>::reference}).
It is equal to \tt{T\&} for real \tt{MatrixView}s, but is
more complicated for complex \tt{MatrixView}s since it needs to 
keep track of the possibility of conjugation.

The main difference between the operator forms and \tt{cref} or \tt{ref} is that the latter versions do not
check for the validity of the parameters \tt{i} and \tt{j}, even when compiling with debugging turned on.
Also, \tt{cref} and \tt{ref} always use \tt{CStyle} indexing.

\item
\begin{tmvcode}
ConstVectorView<T> m.row(int i) const
ConstVectorView<T> m.col(int j) const
VectorView<T> m.row(int i)
VectorView<T> m.col(int j)
\end{tmvcode}
\index{Matrix!methods!row}
\index{Matrix!methods!col}
Return a view of the \tt{i}th row or \tt{j}th column respectively.
If \tt{m} is mutable (either a non-\tt{const Matrix} or a \tt{MatrixView}),
then a \tt{VectorView} is returned.  Otherwise, a \tt{ConstVectorView}
is returned.

\item
\begin{tmvcode}
ConstVectorView<T> m.row(int i, int j1, int j2) const
ConstVectorView<T> m.col(int j, int i1, int i2) const
VectorView<T> m.row(int i, int j1, int j2)
VectorView<T> m.col(int j, int i1, int i2) 
\end{tmvcode}
\index{Matrix!methods!row}
\index{Matrix!methods!col}
Variations on the above, where only a portion of the row or column
is returned.  

For example, with C-style indexing, \tt{m.col(3,2,6)} returns a 4-element
vector view containing the elements
[\tt{m(2,3), m(3,3), m(4,3), m(5,3)}].

With Fortran-style indexing, the same elements are returned by \tt{m.col(4,3,6)}. 
(And these elements would be called: [\tt{m(3,4), m(4,4), m(5,4), m(6,4)}].)

\item
\begin{tmvcode}
ConstVectorView<T> m.diag() const
ConstVectorView<T> m.diag(int i) const 
ConstVectorView<T> m.diag(int i, int k1, int k2) const
VectorView<T> m.diag()
VectorView<T> m.diag(int i)
VectorView<T> m.diag(int i, int k1, int k2)
\end{tmvcode}
\index{Matrix!methods!diag}
Return the diagonal or one of the sub- or super-diagonals.
This first one returns the main diagonal.  For the second and third,
\tt{i=0} refers to the main diagonal; \tt{i>0} are the super-diagonals;
and \tt{i<0} are the sub-diagonals.  The last version is equivalent to the
expression 
\tt{m.diag(i).SubVector(k1,k2)}.
\item
\begin{tmvcode}
ConstVectorView<T> m.SubVector(int i, int j, int istep, int jstep, 
      int size) const
VectorView<T> m.SubVector(int i, int j, int istep, int jstep, int size) 
\end{tmvcode}
\index{Matrix!methods!SubVector}
If the above methods aren't sufficient to obtain the \tt{VectorView} you
need, this function is available, which returns a view through the \tt{Matrix} 
starting at \tt{m(i,j)}, stepping by \tt{(istep,jstep)} between elements,
for a total of \tt{size} elements.  For example, the diagonal
from the lower-left to the upper-right of an $n \times n$ \tt{Matrix} 
would be obtained by: \tt{m.SubVector(n-1,0,-1,1,n)} for C-style or
\tt{m.SubVector(n,1,-1,1,n)} for Fortran-style.

\end{itemize}


\subsection{Views}
\index{Matrix!MatrixView}
\index{Matrix!ConstMatrixView}
\index{Matrix!views}
\label{MViews}

A \tt{MatrixView} object refers to some or all of the elements of a regular \tt{Matrix},
so that altering the elements in the view alters the
corresponding elements in the original object.  A \tt{MatrixView}
can be either row-major, column-major, or neither.  That is, the view can 
span a \tt{Matrix} with non-unit steps in both directions.
It can also be a conjugation of the original
elements.

There are two view classes for
a \tt{Matrix}:
\tt{ConstMatrixView} and \tt{MatrixView}.  
The first is only allowed to view,
not modify, the original elements.  The second is allowed to modify them.

It is worth pointing out again that a 
\tt{const MatrixView} is still mutable, just like a \tt{const VectorView}.
The \tt{const} just means that you cannot change which elements
the view references.

The following methods return views to portions of a \tt{Matrix}.
If \tt{m} is either a (non-\tt{const}) \tt{Matrix}
or a \tt{MatrixView}, then a \tt{MatrixView} is returned.
If \tt{m} is a \tt{const Matrix}, \tt{ConstMatrixView}, or any other \tt{GenMatrix},
then a \tt{ConstMatrixView} will be returned. 

\begin{itemize}
\item
\begin{tmvcode}
m.SubMatrix(int i1, int i2, int j1, int j2)
m.SubMatrix(int i1, int i2, int j1, int j2, int istep, int jstep)
\end{tmvcode}
\index{Matrix!methods!SubMatrix}
This returns a view to a submatrix contained within the original matrix.
\label{matrixaccess}

If \tt{m} uses C-style indexing, 
the upper-left corner of the returned view is \tt{m(i1,j1)},
the lower-left corner is \tt{m(i2-1,j1)},
the upper-right corner is \tt{m(i1,j2-1)}, and
the lower-right corner is \tt{m(i2-1,j2-1)}.

If \tt{m} uses Fortran-style indexing, 
the upper-left corner of the view is \tt{m(i1,j1)},
the lower-left corner is \tt{m(i2,j1)},
the upper-right corner is \tt{m(i1,j2)}, and
the lower-right corner is \tt{m(i2,j2)}.

The second version allows for non-unit steps in the two directions.
To set a \tt{Matrix} to be a checkerboard of 1's, you could write 
(for C-style indexing):
\begin{tmvcode}
tmv::Matrix<int> board(8,8,0)
board.SubMatrix(0,8,0,8,2,2).SetAllTo(1);
board.SubMatrix(1,9,1,9,2,2).SetAllTo(1);
\end{tmvcode}

For Fortran-style indexing, the same thing would be accomplished by:
\begin{tmvcode}
tmv::Matrix<int,tmv::ColMajor,tmv::FortranStyle> board(8,8,0)
board.SubMatrix(1,7,1,7,2,2).SetAllTo(1);
board.SubMatrix(2,8,2,8,2,2).SetAllTo(1);
\end{tmvcode}

\item
\begin{tmvcode}
m.Rows(int i1, int i2)
m.Cols(int j1, int j2)
\end{tmvcode}
\index{Matrix!methods!Rows}
\index{Matrix!methods!Cols}
Since pulling out a bunch of contiguous rows or columns is a common 
submatrix use, we provide these functions.  They are shorthand for
\begin{tmvcode}
m.SubMatrix(i1,i2,0,ncols)
m.SubMatrix(0,nrows,j1,j2)
\end{tmvcode}
respectively.  (For Fortran-style indexing, replace the 0 with a 1.)

\item
\begin{tmvcode}
m.RowPair(i1,i2)
m.ColPair(i1,i2)
\end{tmvcode}
\index{Matrix!methods!RowPair}
\index{Matrix!methods!ColPair}
Another common submatrix is to select a pair of rows or columns, not 
necessarily adjacent to each other.  These are short hand for:
\begin{tmvcode}
m.SubMatrix(i1,i2+(i2-i1),0,ncols,i2-i1,1)
m.SubMatrix(0,nrows,j1,j2+(j2-j1),1,j2-j1)
\end{tmvcode}
respectively.  The equivalent in Fortran-style indexing would be:
\begin{tmvcode}
m.SubMatrix(i1,i2,1,ncols,i2-i1,1)
m.SubMatrix(1,nrows,j1,j2,1,j2-j1).
\end{tmvcode}

\item
\begin{tmvcode}
m.Transpose()
m.Conjugate()
m.Adjoint()
\end{tmvcode}
\index{Matrix!methods!Transpose}
\index{Matrix!methods!Conjugate}
\index{Matrix!methods!Adjoint}
These return the transpose, conjugate, and adjoint (aka conjugate-transpose) 
of a \tt{Matrix}.  They point to the 
same physical elements as the original matrix, so modifying these will
correspondingly modify the original matrix.

Note that some people define the adjoint of a matrix as the determinant times
the inverse.  This combination is also called the adjugate or the cofactor matrix.
It is \underline{not} the same as our \tt{m.Adjoint()}.  What we call the adjoint 
is usually written as $m^\dagger$, or variously as $m^H$ or $m^*$, 
and is sometimes referred to as the hermitian conjugate
or (rarely) tranjugate.  This definition of the adjoint seems to be the more modern
usage.  Older texts tend to use the other definition.  However, if this is confusing
for you, it may be clearer to explicitly write out \tt{m.Conjugate().Transpose()},
which will not produce any efficiency reduction in your code compared with using
\tt{m.Adjoint()} (assuming your compiler inlines these methods properly).

\item
\begin{tmvcode}
m.View()
\end{tmvcode}
\index{Matrix!methods!View}
Returns a view of a \tt{Matrix}.  As with the \tt{View()} function for a \tt{Vector}, it is mostly
useful for passing a \tt{Matrix} to a function that takes a \tt{MatrixView} argument.  
This lets you convert the first into the second.

\item
\begin{tmvcode}
m.Real()
m.Imag()
\end{tmvcode}
\index{Matrix!methods!Real}
\index{Matrix!methods!Imag}
These return views to the real and imaginary parts of a complex \tt{Matrix}.
Note the return type is a real view in each case:
\begin{tmvcode}
tmv::Matrix<std::complex<double> > m(10,std::complex<double>(1,4));
tmv::MatrixView<double> mr = m.Real();
tmv::MatrixView<double> mi = m.Imag();
\end{tmvcode}

\item
\begin{tmvcode}
m.UpperTri(DiagType dt = NonUnitDiag)
m.LowerTri(DiagType dt = NonUnitDiag)
\end{tmvcode}
\index{Matrix!methods!UpperTri}
\index{Matrix!methods!LowerTri}
These return an \tt{UpperTriMatrixView} or a \tt{LowerTriMatrixView} which
views either the upper triangle or the lower triangle of a square \tt{Matrix}.
If \tt{m} has more rows than columns, then only \tt{UpperTri} is valid, since
the portion below the diagonal is not triangular.
Likewise, if \tt{m} has more columns than rows, then only \tt{LowerTri} is valid.

In both cases, you may provide an optional parameter \tt{dt}, which
declares whether the diagonal elements are treated as all $1$s
(\tt{dt = UnitDiag}) or as their actual values (\tt{dt = NonUnitDiag}). 
See \S\ref{trimatrix} for more details about this parameter and
triangular matrices in general.

\end{itemize}

\subsection{Functions of a matrix}
\index{Matrix!functions of}
\label{matrixfunctions}

Functions that do not modify the \tt{Matrix} are defined in 
\tt{GenMatrix}, and so can be used for any type derived from \tt{GenMatrix}:
\tt{Matrix}, \tt{ConstMatrixView}, \tt{MatrixView}, or \tt{MatrixComposite}.
Functions that modify the \tt{Matrix} are only defined for 
\tt{Matrix} and \tt{MatrixView}.

\subsubsection{Non-modifying functions}

Each of the following functions can be written in two ways,
either as a method or a function.
For example, the expressions \tt{m.Norm()} and \tt{Norm(m)}
are equivalent.  
In each case, \tt{m} can be any \tt{GenMatrix<T>}.
As a reminder, \tt{RT} refers to the real type associated with \tt{T}.
In other words, \tt{T} is either the same as \tt{RT} or it is
\tt{std::complex<RT>}.

\begin{itemize}

\item
\begin{tmvcode}
RT m.Norm1() const
RT Norm1(m)
\end{tmvcode}
\index{Matrix!methods!Norm1}
\index{Matrix!functions of!Norm1}
The 1-norm of \tt{m}: 
$||m||_1 = \max_j (\sum_i |m(i,j)|)$.

\item
\begin{tmvcode}
RT m.Norm2() const
RT Norm2(m)
RT m.DoNorm2() const
\end{tmvcode}
\index{Matrix!methods!Norm2}
\index{Matrix!methods!DoNorm2}
\index{Matrix!functions of!Norm2}
The 2-norm of \tt{m}: $||m||_2 =$ the largest singular value of $m$, which
is also the square root of the largest eigenvalue of $(m^\dagger m)$.

This function can be fairly expensive if you have not
already performed an SV decomposition of \tt{m}, so the first two versions
output a warning to \tt{stdout} if there is no SV decomposition already set.

If you understand that you are asking TMV to perform an SV decomposition
to calculate $||m||_2$, and you are ok with it, you can either change the warning
behavior -- see \S\ref{warnings} for how to do this --
or you can just call \tt{DoNorm2()} instead, which will do the 
necessary SV decomposition without any warnings.

In fact, if you are not going to need the SVD for anything else, then 
\tt{m.DoNorm2()} will be faster than \tt{m.SVD(), m.Norm2()}, because the 
former will not bother to accumulate the singular vectors.

\item
\begin{tmvcode}
RT m.NormInf() const
RT NormInf(m)
\end{tmvcode}
\index{Matrix!methods!NormInf}
\index{Matrix!functions of!NormInf}
The infinity-norm of \tt{m}: 
$||m||_\infty = \max_i (\sum_j |m(i,j)|)$.

\item
\begin{tmvcode}
RT m.NormF() const
RT NormF(m)
RT m.Norm() const
RT Norm(m)
\end{tmvcode}
\index{Matrix!methods!Norm}
\index{Matrix!functions of!Norm}
\index{Matrix!methods!NormF}
\index{Matrix!functions of!NormF}
The Frobenius norm of \tt{m}: 
$||m||_F = (\sum_{i,j} |m(i,j)|^2)^{1/2}$.

This is the most common meaning for the norm of a matrix, so we
define the \tt{Norm} function to be the same as \tt{NormF}.

\item
\begin{tmvcode}
RT m.NormSq(RT scale=1) const
RT NormSq(m)
\end{tmvcode}
\index{Matrix!methods!NormSq}
\index{Matrix!functions of!NormSq}
The square of the Frobenius norm of \tt{m}: 
$(||m||_F)^2 = \sum_{i,j} |m(i,j)|^2$.

In the method version of this function, you may provide an optional scale factor,
in which case the return value is equal to NormSq(scale*v) instead, 
which can help avoid underflow or overflow problems.

\item
\begin{tmvcode}
RT m.MaxAbsElement() const
RT MaxAbsElement(m)
\end{tmvcode}
\index{Matrix!methods!MaxAbsElement}
\index{Matrix!functions of!MaxAbsElement}
The element of \tt{m} with the maximum absolute value: 
$||m||_\Delta = \max_{i,j} |m(i,j)|$.

\item
\begin{tmvcode}
T m.Trace() const
T Trace(m)
\end{tmvcode}
\index{Matrix!methods!Trace}
\index{Matrix!functions of!Trace}
The trace of \tt{m}: $\mathrm{Tr}(m) = \sum_i m(i,i)$.

\item
\begin{tmvcode}
T m.Det() const
T Det(m)
\end{tmvcode}
\index{Matrix!determinant}
\index{Matrix!methods!Det}
\index{Matrix!functions of!Det}
The determinant of \tt{m}, $\det(m)$.  For speed issues regarding this function, see 
\S\ref{division} below on division.

\item
\begin{tmvcode}
RT m.LogDet(T* sign=0) const
RT LogDet(m)
\end{tmvcode}
\index{Matrix!determinant}
\index{Matrix!methods!LogDet}
\index{Matrix!functions of!LogDet}
The log of the absolute value of the determinant of \tt{m}.  If the optional argument \tt{sign} is 
provided (possible in the method version only), then on output \tt{*sign} records the sign of the determinant.  See \ref{determinants} 
for more details.

\item
\begin{tmvcode}
bool m.Singular() const
\end{tmvcode}
\index{Matrix!methods!Singular}
Return whether \tt{m} is singular, i.e. $\det(m) = 0$.
(Singular matrices are discussed in more detail in \S\ref{singular}.)

\item
\begin{tmvcode}
RT m.Condition() const
RT m.DoCondition() const
\end{tmvcode}
\index{Matrix!methods!Condition}
\index{Matrix!methods!DoCondition}
The condition (technically the 2-condition) is 
the ratio of the largest singular value of $m$ to the smallest.

Like \tt{Norm2}, this function requires a singular value decomposition to be performed,
so it can be fairly expensive if you have not
already performed an SV decomposition of \tt{m}.
So the first version
outputs a warning to \tt{stdout} if there is no SV decomposition already set for the matrix.

And as with \tt{Norm2}, you can bypass the warning by calling \tt{DoCondition()} instead,
which will do the necessary SV decomposition without any warnings.  
Also, see \S\ref{warnings} for how to change the warning behavior of TMV.

\item
\begin{tmvcode}
tmv::Matrix<T> minv = m.Inverse()
tmv::Matrix<T> minv = Inverse(m)
void m.Inverse(Matrix<T>& minv)
\end{tmvcode}
\index{Matrix!methods!Inverse}
\index{Matrix!functions of!Inverse}
Set \tt{minv} to the inverse of \tt{m}.  

If \tt{m} is not square, then \tt{minv} is set to the pseudo-inverse, or an approximate
pseudo-inverse.  If \tt{m} is singular, then an error may result, or the pseudo-inverse
may be returned, depending on the division method specified for the matrix.  
See \S\ref{pseudoinverse} and \S\ref{singular} 
on pseudo-inverses and singular matrices for more details.

Note that the first two forms do not actually require a 
temporary (despite appearances), so they are just as efficient as the third version.
This is because \tt{m.Inverse()} actually returns an object whose type is derived from
\tt{MatrixComposite}.  The calculation of the inverse is then delayed 
until there is a place to store the result.

\item
\begin{tmvcode}
void m.InverseATA(Matrix<T>& cov) const
\end{tmvcode}
\index{Matrix!methods!InverseATA}
Set \tt{cov} to be $(m^\dagger m)^{-1}$.

If \tt{m} has more rows than columns, then using it to solve a system of equations
really amounts to finding the least-square solution, since there is (typically) no
exact solution.  When you do this, \tt{m} is known as the ``design matrix'' of the system,
and is commonly called $A$.  Solving $A x=b$ gives $x$ as the least-square 
solution.  And the covariance matrix for the solution vector $x$ is 
$\Sigma = (A^\dagger A)^{-1}$.
It turns out that computing this matrix is generally easy to do once you have 
performed the decomposition needed to solve for $x$ (either a QR or SV 
decomposition - see \S\ref{divtypes}).  
Thus this function is provided, which sets the
argument \tt{cov} to the equivalent of \tt{Inverse(m.Adjoint()*m)}, 
but generally does so much more efficiently than doing this explicitly, 
and also probably more accurately.

\end{itemize}

\subsubsection{Modifying functions}

The following functions are methods of both \tt{Matrix} and \tt{MatrixView},
and they work the same way for each.
As with the \tt{Vector} modifying functions, these all return a reference
to the newly modified \tt{Matrix}, so you can string them together if you want.

\begin{itemize}

\item
\begin{tmvcode}
m.Zero()
\end{tmvcode}
\index{Matrix!methods!Zero}
Set all elements to 0.

\item
\begin{tmvcode}
m.SetAllTo(T x)
\end{tmvcode}
\index{Matrix!methods!SetAllTo}
Set all elements to the value \tt{x}.

\item
\begin{tmvcode}
m.Clip(RT thresh)
\end{tmvcode}
\index{Matrix!methods!Clip}
Set each element whose absolute value is less than \tt{thresh} equal to 0.
Note that \tt{thresh} should be a real value even for complex valued
\tt{Matrix}es.

\item
\begin{tmvcode}
m.SetToIdentity(T x = 1)
\end{tmvcode}
\index{Matrix!methods!SetToIdentity}
Set \tt{m} to \tt{x} times the identity matrix.
If the argument \tt{x} is omitted, it is 
taken to be \tt{1}, so \tt{m} is set to the identity matrix.
This is equivalent to \tt{m.Zero().diag().SetAllTo(x)}.

\item 
\begin{tmvcode}
m.ConjugateSelf()
\end{tmvcode}
\index{Matrix!methods!ConjugateSelf}
Conjugate each element.  Note the difference between this and \tt{m.Conjugate()}, 
which returns a \underline{view} to the conjugate of \tt{m} without
actually changing the underlying data.  Contrariwise, \tt{m.ConjugateSelf()}
does change the underlying data.

\item
\begin{tmvcode}
m.TransposeSelf()
\end{tmvcode}
\index{Matrix!methods!TransposeSelf}
Transpose the \tt{Matrix}.  Note the difference between this and 
\tt{m.Transpose()}, which returns a \underline{view} to the transpose without 
actually changing the underlying data.

\item
\begin{tmvcode}
m.SwapRows(int i1, int i2)
m.SwapCols(int j1, int j2)
\end{tmvcode}
\index{Matrix!methods!SwapRows}
\index{Matrix!methods!SwapCols}
Swap the corresponding elements of two rows or two columns.

\item
\begin{tmvcode}
m.PermuteRows(const int* p)
m.PermuteRows(const int* p, int i1, int i2)
m.ReversePermuteRows(const int* p)
m.ReversePermuteRows(const int* p, int i1, int i2)
\end{tmvcode}
\index{Matrix!methods!PermuteRows}
\index{Matrix!methods!ReversePermuteRows}
These are equivalent to the corresponding routines for \tt{Vector}s 
(\tt{Permute} and \tt{ReversePermute}), performing a series of \tt{SwapRows} commands.

\item
\begin{tmvcode}
m.PermuteCols(const int* p)
m.PermuteCols(const int* p, int j1, int j2)
m.ReversePermuteCols(const int* p)
m.ReversePermuteCols(const int* p, int j1, int j2)
\end{tmvcode}
\index{Matrix!methods!PermuteCols}
\index{Matrix!methods!ReversePermuteCols}
Same as above, but performing the a series of \tt{SwapCols} commands.

\item
\begin{tmvcode}
Swap(m1,m2)
\end{tmvcode}
\index{Matrix!functions of!Swap}
Swap the corresponding elements of \tt{m1} and \tt{m2}.  Note that this does physically
swap the data elements, not just some pointers to the data, so it takes $O(N^2)$ time.

\end{itemize}

\subsection{Arithmetic}
\index{Matrix!arithmetic}

\subsubsection{Basic Operators}

We'll start with the simple operators that require little explanation aside from the 
notation: 
\tt{x} is a scalar, \tt{v} is a \tt{Vector}, and \tt{m} is a \tt{Matrix}.  As a reminder,
the notation \tt{[+-]} is used to indicate either \tt{+} or \tt{-}.  Likewise for \tt{[*/]}.
\begin{tmvcode}
m2 = -m1
m2 = x * m1
m2 = m1 [*/] x
m3 = m1 [+-] m2
m [*/]= x
m2 [+-]= m1
v2 = m * v1
v2 = v1 * m
v *= m
m3 = m1 * m2
m2 *= m1
\end{tmvcode}

Note that the orientation of a vector is inferred from context.  \tt{m*v} involves a 
column vector, and \tt{v*m} involves a row vector.

It is sometimes convenient to be able to treat scalars as the scalar times an
identity matrix.  So the following operations are allowed and use that convention:
\begin{tmvcode}
m2 = m1 [+-] x
m2 = x [+-] m1
m [+-]= x
m = x
\end{tmvcode}
For example, you could check if a matrix is numerically close to the identity matrix 
with:
\begin{tmvcode}
if (Norm(m-1.) < 1.e-8) { [...] }
\end{tmvcode}

\subsubsection{Outer Products}

For the product of two vectors, there are two orientation choices that make sense mathematically.
It could mean a row vector times a column vector, which is called the inner product.
Or it could mean a column vector times a row vector, which is called the outer product.
We chose to let \tt{v1*v2} indicate the inner product, since this is far more common.
For the outer product, we use a different symbol:
\begin{tmvcode}
m = v1 ^ v2
\end{tmvcode}
\index{Vector!arithmetic!outer product}
\index{Matrix!arithmetic!rank-1 update}
One problem with this choice is that the order of operations in C++ for \tt{\^} 
is not the same as for \tt{*}.  
So, one needs to be careful when combining it with other calculations.
For example
\begin{tmvcode}
m2 = m1 + v1 ^ v2   // ERROR!
\end{tmvcode}
will give a compile time error indicating that you can't add a 
\tt{Matrix} and a \tt{Vector}, 
because the operator \tt{+} has higher precedence in C++ than \tt{\^}.
So you need to write:
\begin{tmvcode}
m2 = m1 + (v1 ^ v2)
\end{tmvcode}

\subsubsection{Subroutines}

As with \tt{Vector}s, we try to defer calculations until we have a place to store them.
So \tt{m*v} returns an object that can be assigned to a \tt{Vector},
but hasn't performed the calculation yet.  

The limit to how complicated an expression can be without resorting to a temporary object
is set by the 
functions that the code eventually calls to perform the calculation.  
While you shouldn't ever
need to use these directly, it may help you understand when the code will create a
temporary \tt{Matrix}\footnote{If you do use these, note that the last parameters are
\tt{MatrixView}s, rather than \tt{Matrix}es.  So you would need to 
call them with \tt{m.View()} if \tt{m} is a \tt{Matrix}.  (For \tt{MultMV},
it would be \tt{v.View()}.)}.

\begin{itemize}

\item
\begin{tmvcode}
MultXM(T x, const MatrixView<T>& m)
\end{tmvcode}
\index{Matrix!arithmetic!MultXM function}
Performs the calculation \tt{m *= x}.

\item
\begin{tmvcode}
MultMV<bool add>(T x, const GenMatrix<Tm>& m, 
      const GenVector<Tv1>& v1, const VectorView<T>& v2)
\end{tmvcode}
\index{Matrix!arithmetic!MultMV function}
Performs the calculation \tt{v2 (+=) x*m*v1} 
where ``\tt{(+=)}'' means ``\tt{+=}'' 
if \tt{add} is true and ``\tt{=}'' if \tt{add} is false.

\item
\begin{tmvcode}
AddMM(T x, const GenMatrix<T1>& m1, const MatrixView<T>& m2)
\end{tmvcode}
\index{Matrix!arithmetic!AddMM function}
Performs the calculation \tt{m2 += x*m1}.

\item
\begin{tmvcode}
AddMM(T x1, const GenMatrix<T1>& m1, T x2, const GenMatrix<T2>& m2,
      const MatrixView<T>& m3)
\end{tmvcode}
\index{Matrix!arithmetic!AddMM function}
Performs the calculation \tt{m3 = x1*m1 + x2*m2}.

\item
\begin{tmvcode}
MultMM<bool add>(T x, const GenMatrix<T1>& m1, 
      const GenMatrix<T2>& m2, const MatrixView<T>& m3)
\end{tmvcode}
\index{Matrix!arithmetic!MultMM function}
Performs the calculation \tt{m3 (+=) x*m1*m2} 
where ``\tt{(+=)}'' means ``\tt{+=}'' 
if \tt{add} is true and ``\tt{=}'' if \tt{add} is false.

\item
\begin{tmvcode}
Rank1Update<bool add>(T x, const GenVector<T1>& v1, 
      const GenVector<T2>& v2, const MatrixView<T>& m)
\end{tmvcode}
\index{Matrix!arithmetic!Rank1Update function}
\index{Matrix!arithmetic!rank-1 update}
Performs the calculation 
%\tt{m (+=) x*(v1\caret v2)} 
\tt{m (+=) x*(v1 ^ v2)} 
where ``\tt{(+=)}'' means ``\tt{+=}'' 
if \tt{add} is true and ``\tt{=}'' if \tt{add} is false.

\end{itemize}

\subsection{Matrix division}
\index{Matrix!arithmetic!division}
\index{Linear equations!exact solution}
\label{division}

One of the main things people often want to do with a matrix is use it to solve a 
set of linear equations.  The set of equations can be written as a single matrix
equation:
\begin{equation*}
\nonumber
A x = b
\end{equation*}
where $A$ is a matrix and $x$ and $b$ are vectors.  $A$ and $b$ are known, 
and one wants to solve for $x$.  Sometimes there are
multiple systems to be solved using the same coefficients, in which case
$x$ and $b$ become matrices as well.

\subsubsection{Operators}
\label{divops}

Using the TMV classes, one would solve this equations by writing simply:
\begin{tmvcode}
x = b / A
\end{tmvcode}
Note that this really means $x = A^{-1} b$, which is different from $x = b A^{-1}$.
Writing the matrix equation as we did ($A x=b$) is much more common than 
writing $xA=b$, so ``left-division'' is correspondingly much more common than 
``right-division''.  Therefore, it makes sense to use left-division for our definition of the \tt{/}
operator.

However, we do allow for the possibility of wanting to right-multiply a vector
by $A^{-1}$ (in which case the vector is inferred to be a row-vector).  We designate
this operation by:
\begin{tmvcode}
x = b % A
\end{tmvcode}
which means $x = b A^{-1}$.

Given this explanation, the rest of the division operations should be self-explanatory,
where we use the notation \tt{[/\%]} to indicate that either \tt{/} or \tt{\%} may
be used with the above difference in meaning:
\begin{tmvcode}
v2 = v1 [/%] m
m3 = m1 [/%] m2
v [/%]= m
m2 [/%]= m1
m2 = x [/%] m1
\end{tmvcode}

If you feel uncomfortable using the \tt{/} and \tt{\%} symbols,
you can also explicitly write things like
\begin{tmvcode}
v2 = m.Inverse() * v1
v3 = v1 * m.Inverse()
\end{tmvcode}
which delay the calculation in exactly the same way that the above forms do.  
These forms
do not ever explicitly calculate the matrix inverse, since this is not (numerically) a
good way to perform these calculations.  Instead, the appropriate decomposition 
(see \S\ref{divtypes})
is used to calculate \tt{v2} and \tt{v3}.

\subsubsection{Least-square solutions}
\index{Matrix!arithmetic!division}
\index{Linear equations!least square solution}
\label{leastsquare}

If $A$ is not square, then the equation $A x = b$ does not have a unique solution.
If $A$ has more rows than columns, then there is in general no solution.
And if $A$ has more columns than rows, then there are an infinite 
number of solutions.  

The former case is more common and represents an overdetermined system of 
equations.  In this case, one is not looking for an exact solution
for $x$, but rather the value of $x$ that minimizes $||b - A x||_2$.  This is the 
meaning of the least-square
solution, and is the value returned by \tt{x = b/A} for the TMV classes.  

The matrix $A$ is called the ``design'' matrix.  For example, assume you are doing
a simple quadratic fit to some data $(x_i,y_i)$, and the model your are fitting for
can be written as
$y = c + dx + ex^2$.  Furthermore, assume that each
$y_i$ value has a measurement error of $s_i$.  Then the rows of $A$ should
be set to: $( ~1/s_i \quad x_i/s_i \quad x_i^2/s_i ~ )$.  The corresponding
element of the vector $b$ shoudl be $( ~ y_i/s_i ~ )$.  It is easily verified that
$||b-Ax||_2^2$ is the normal $\chi^2$ expression.
The solution returned by
\tt{x = b/A} would then be the least-square fit solution: $(~c \quad d \quad e~)$,
which would be the solution which minimizes $\chi^2$.

The underdetermined case is not so common, but can still be defined reasonably.  
As mentioned above, there are infinitely many solutions to such an equation, so the
value returned by \tt{x = b/A} in this case is the value of \tt{x} that satisfies the 
equation and has minimum 2-norm, $||x||_2$.

When you have calculated a least-square solution for \tt{x}, it is common to want 
to know the covariance matrix of the returned values.  It turns out that this
matrix is $(A^\dagger A)^{-1}$.  It is not very efficient to calculate this matrix
explicitly and then invert it.  But once you have calculated the decomposition
needed for the division, it is quite easy.  So we provide the routine 
\begin{tmvcode}
A.InverseATA(Matrix<T>& cov)
\end{tmvcode}
to perform the calculation efficiently.  (Make sure you save the decomposition with 
\tt{A.SaveDiv()} - see \S\ref{efficiency} for more about this.)

\subsubsection{Decompositions}
\index{Matrix!decompositions}
\index{Matrix!arithmetic!division}
\label{divtypes}

There are quite a few ways to go about solving the equations written above.  
The more efficient ways involve decomposing $A$ into a product of
matrices with special structures or properties.  
You can select which decomposition to use with the method:
\begin{tmvcode}
m.DivideUsing(tmv::DivType dt)
\end{tmvcode}
where \tt{dt} can be any of \tt{\{tmv::LU, tmv::QR, tmv::QRP, tmv::SV\}}.
If you do not specify which decomposition to use, \tt{LU} is the 
default for square matrices, and \tt{QR} is the default for
non-square matrices.

\begin{enumerate}
\item
\textbf{LU Decomposition}: 
\index{Matrix!LU decomposition}
\index{LU Decomposition!Matrix}
(\tt{dt} = \tt{tmv::LU}) $A = P L U$, where $L$ is a lower-triangle 
matrix with all 1's along the diagonal, $U$ is an upper-triangle matrix, 
and $P$ is a permutation matrix.  This decomposition is only available for 
square matrices.

\item
\textbf{QR Decomposition}: 
\index{Matrix!QR decomposition}
\index{QR Decomposition!Matrix}
(\tt{dt} = \tt{tmv::QR}) $A = Q R$ where $Q$ is a unitary matrix
and $R$ is an upper-triangle matrix.  (Note: real unitary matrices are
also known as orthogonal matrices.)  A unitary matrix is such that
$Q^\dagger Q = I$. 

If $A$ has dimensions $M \times N$ with $M > N$, 
then $R$ has dimensions $N \times N$, and $Q$
has dimensions $M \times N$.  In this case, $Q$ will
only be column-unitary.  That is $Q Q^\dagger \neq I$.

If $M < N$, then $A^T$ is actually decomposed into $Q R$.

\item
\textbf{QRP Decomposition}: 
\index{Matrix!QRP decomposition}
\index{QRP Decomposition!Matrix}
(\tt{dt} = \tt{tmv::QRP}) $A = Q R P$ where $Q$ is unitary, $R$ is 
upper-triangle, and $P$ is a permutation.  This decomposition is somewhat
slower than a simple QR decomposition, but it is numerically more stable if
$A$ is singular, or nearly so.  
(Singular matrices are discussed in more detail in \S\ref{singular}.)

There are two slightly different algorithms for doing a QRP decomposition, controlled by a global
\tt{bool} variable: \tt{tmv::QRPDiv<T>::StrictQRP}.
If this is set to true, then the decomposition will make the diagonal elements
of $R$ be strictly decreasing (in absolute value) from upper-left to lower-right.

If it is false, however (the default), then there will be no diagonal element
of $R$ below and to the right of one which is \underline{much} smaller in absolute value,
where ``much'' means the ratio will be at most $\epsilon^{1/4}$, where 
$\epsilon$ is the machine precision for the type in question.  This restriction
is almost always sufficient to make the decomposition useful for singular or nearly
singular matrices, and it is much faster than the strict algorithm.

\item
\textbf{Singular Value Decomposition}: 
\index{Matrix!singular value decomposition}
\index{Singular Value Decomposition!Matrix}
(\tt{dt} = \tt{tmv::SV}) $A = U S V$ where $U$ is unitary
(or column-unitary if $M > N$), $S$ is diagonal and real, and $V$ is unitary 
(or row-unitary if $M < N$).  The values of $S$ will be such that all the values will
be non-negative and will decrease along the diagonal.
The singular value decomposition is most useful
for matrices that are singular or nearly so.  
We will discuss this decomposition in more detail in \S\ref{singular} on
singular matrices below.

\end{enumerate}

\subsubsection{Pseudo-inverse}
\index{Matrix!pseudo-inverse}
\label{pseudoinverse}

If \tt{m} is not square, then \tt{m.Inverse()} should return what is called the pseudo-inverse.
If \tt{m} has more rows than columns, then \tt{m.Inverse() * m} is the identity matrix, 
but \tt{m * m.Inverse()}
is not an identity.  If \tt{m} has more columns than rows, then the opposite holds.

Here are some features of the pseudo-inverse (we use $X$ to represent the pseudo-inverse
of $M$):
\begin{align*}
M X M &= M \\
X M X &= X \\
(M X)^T &= M X \\
(X M)^T &= X M 
\end{align*}
For singular square matrices, 
one can also define a pseudo-inverse with the same properties.

In the first sentence of this section, I used the word ``should''.  
This is because the different decompositions calculate the pseudo-inverse 
differently and result in slightly
different answers.  For QR or QRP, the matrix returned by \tt{m.Inverse()} for
non-square matrices isn't
quite correct.  When $M$ is not square, but is also not singular, then $X$
will satisfy the first three of the above equations,
but not the last one.  With the SV decomposition, however,
$X$ is the true pseudo-inverse and all four equations are satisfied.

For singular matrices, QR will fail to give a good pseudo-inverse (and may throw the
\tt{tmv::Singular} exception - c.f. \S\ref{error}), QRP will be close to correct
(again failing only the last equation) and will not throw an exception, and SV will be correct.
\index{Exceptions!Singular}

\subsubsection{Efficiency issues}
\label{efficiency}
\index{Matrix!arithmetic!division}

Let's say you compute a matrix decomposition for a particular division calculation, and
then later want to use it again for a different right hand side:
\begin{tmvcode}
x = b / m;
[...]
y = c / m;
\end{tmvcode}
Ideally, the code would just use the same decomposition that had already been calculated
for the \tt{x} assignment when it gets to the later assignment of \tt{y}, 
so this second division would be very fast.
However, what if somewhere in the \tt{[...]}, the matrix \tt{m} is modified?  
Then using
the same decomposition would be incorrect - a new decomposition would be needed
for the new matrix values.

One solution might be to try to keep track of when a \tt{Matrix} gets changed.  
We could set an
internal flag whenever it is changed to indicate that
any existing decomposition is invalid.  While not impossible,
this type of check is made
quite difficult by the way we have allowed different view objects to point to the same data.  
It would
be difficult, and probably very slow, to make sure that any change in one view invalidates the decompositions
of all other views to the same data.

Our solution is instead to err on the side of correctness over efficiency and
to always recalculate the decomposition by default.  Of course, this
can be quite inefficient, so we allow the programmer to override this
behavior for a specific \tt{Matrix} object with the method:
\begin{tmvcode}
m.SaveDiv()
\end{tmvcode}
\index{Matrix!methods!SaveDiv}
After this call, whenever a decomposition is set, it is saved for any future uses.
You are telling the program to assume that the values of \tt{m} will not change after 
that point (technically after
the next decomposition is calculated).
\label{matrixsetdiv}

If you do modify \tt{m} after a call to \tt{SaveDiv()}, 
you can manually reset the decomposition with
\begin{tmvcode}
m.ReSetDiv()
\end{tmvcode}
\index{Matrix!methods!ReSetDiv}
which deletes any current saved decomposition, and recalculates it.
Similarly,
\begin{tmvcode}
m.UnSetDiv()
\end{tmvcode}
\index{Matrix!methods!UnSetDiv}
will delete any current saved decomposition, but not calculate a new one.
This can be used to free up the memory that the decomposition had been using.

Sometimes you may want to set a decomposition before you actually need it.
For example, the division may be in a speed critical part of the code, but you have
access to the \tt{Matrix} well before then.  
You can tell the object to calculate the decomposition with
\begin{tmvcode}
m.SetDiv()
\end{tmvcode}
\index{Matrix!methods!SetDiv}
This may also be useful if you just want to perform and access the decomposition
separate from any actual division statement (e.g. SVD for principal component analysis).
You can also determine whether the decomposition has be set yet with:
\begin{tmvcode}
bool m.DivIsSet()
\end{tmvcode}
\index{Matrix!methods!DivIsSet}

Also, if you change what kind of decomposition the \tt{Matrix} should use by calling
\tt{DivideUsing(...)}, then this will also delete any existing decomposition that 
might be saved (unless you ``change'' it to the same thing).

Finally, there is another efficiency issue, which can sometimes be important.  The default
behavior is to use extra memory for calculating the decomposition, so the original
matrix is left unchanged.  However, it is often the case that once you have 
calculated the decomposition, you don't need the original matrix anymore.
In that case, it is ok to overwrite the original matrix.  For very large matrices, 
the savings in memory may be significant.  (The $O(N^2)$ steps in copying 
the \tt{Matrix} is generally negligible compared to the $O(N^3)$ steps in performing
the decomposition.  So memory issues are probably the only reason to do this.)

Therefore, we provide another routine that lets the decomposition be 
calculated in place, overwriting the original \tt{Matrix}\footnote{
Note: for a regular \tt{Matrix}, this is always possible.  However, for some of the 
special matrix varieties, there are decompositions which cannot be done in place.
Whenever that is the case, this directive will be ignored.}:
\begin{tmvcode}
m.DivideInPlace()
\end{tmvcode}
\index{Matrix!methods!DivideInPlace}

\subsubsection{Determinants}
\index{Matrix!determinant}
\index{Determinant}
\label{determinants}
\index{Matrix!methods!Det}
\index{Matrix!methods!LogDet}

Aside from very small matrices, the calculation of the determinant typically 
requires calculations similar to those performed in
the above decompositions.  Since a determinant only really makes sense
for square matrices, one would typically perform an LU decomposition to calculate
the determinant.  Then the determinant of $A$ is just the determinant of $U$
(possibly times $-1$ depending on the details of $P$), which in turn is simply the 
product of the values along the diagonal.

Therefore, calling \tt{m.Det()} involves calculating the LU decomposition, and then
finding the determinant of $U$.  
If you are also performing a division calculation, you should probably use
\tt{m.SaveDiv()} to avoid calculating the decomposition twice.

If you have set \tt{m} to use some other decomposition using 
\tt{m.DivideUsing(...)},
then the determinant will be determined from that decomposition instead 
(which is always similarly easy).

For large matrices, the value of the determinant can be extremely large, which can easily
lead to overflow problems, even for only moderately large matrices.  Therefore, we
also provide the method \tt{m.LogDet()} which calculates the natural logarithm of the
absolute value of the determinant.  This method can be given an argument, \tt{sign},
which returns the sign of the determinant.  The actual determinant can then be 
reconstructed from 
\begin{tmvcode}
T sign;
T logdet = m.LogDet(&sign);
T det = sign * exp(logdet);
\end{tmvcode}
If \tt{m} is a complex matrix, then the ``sign'' is really a complex number whose 
absolute vale is $1$. It is defined to be the value of \tt{(det/abs(det))}.

This alternate calculation is especially useful for non-linear maximum likelihood
calculations.  Since log is a monotonic function, the maximum likelihood is coincident
with the maximum of its logarithm.  And, since likelihoods in matrix formulation
often involve a determinant,
the \tt{LogDet()} function is exactly what is needed to calculate the log likelihood.

\subsubsection{Accessing the decompositions}
\index{Matrix!decompositions}

Sometimes, you may want to access the components of the decomposition directly,
rather than just use them for performing the division or calculating the determinant.

\begin{itemize}
\item
For the LU decomposition, we have:
\begin{tmvcode}
tmv::ConstLowerTriMatrixView<T> m.LUD().GetL()
tmv::ConstUpperTriMatrixView<T> m.LUD().GetU()
int* m.LUD().GetP()
bool m.LUD().IsTrans()
\end{tmvcode}
\index{Matrix!methods!LUD}
\index{Matrix!LU decomposition}
\index{LU Decomposition!Matrix}
\tt{GetL()} and \tt{GetU()} return $L$ and $U$.
\tt{GetP()} returns the permutation array, $P$, in a form
that can be used as an argument 
to the routines like \tt{m.PermuteRows(P)}.  Finally, \tt{IsTrans()} returns whether
the product $PLU$ is equal to \tt{m} or \tt{m.Transpose()}.  

The following should result in a \tt{Matrix m2}, which is numerically very close to
the original \tt{Matrix m}:
\begin{tmvcode}
tmv::Matrix<T> m2 = m.LUD().GetL() * m.LUD().GetU();
m2.ReversePermuteRows(m.LUD().GetP());
if (m.LUD().IsTrans()) m2.TransposeSelf();
\end{tmvcode}

\item
For the QR decomposition, we have:
\begin{tmvcode}
tmv::PackedQ<T> m.QRD().GetQ();
tmv::ConstUpperTriMatrix<T> m.QRD().GetR();
bool m.QRD().IsTrans();
\end{tmvcode}
\index{Matrix!methods!QRD}
\index{Matrix!QR decomposition}
\index{QR Decomposition!Matrix}
\tt{GetQ()} and \tt{GetR()} return $Q$ and $R$.  
\tt{IsTrans()} returns whether
the decomposition is equal to \tt{m} or \tt{m.Transpose()}.  

Note: the \tt{PackedQ} class is convertible into a regular \tt{Matrix}, but if you are doing 
arithmetic with it, then these can generally be done without any conversion,
so it is more efficient.

The following should result in a \tt{Matrix m2}, which is numerically very close to
the original \tt{Matrix m}:
\begin{tmvcode}
tmv::Matrix<T> m2(m.nrows(),m.ncols());
tmv::MatrixView<T> m2v = 
      m.QRD().IsTrans() ? m2.Transpose() : m2.View();
m2v = m.QRD().GetQ() * m.QRD().GetR();
\end{tmvcode}

\item
For the QRP decomposition, we have:
\begin{tmvcode}
tmv::PackedQ<T> m.QRPD().GetQ()
tmv::ConstUpperTriMatrixView<T> m.QRPD().GetR()
int* m.QRPD().GetP()
bool m.QRPD().IsTrans()
\end{tmvcode}
\index{Matrix!methods!QRPD}
\index{Matrix!QRP decomposition}
\index{QRP Decomposition!Matrix}
\tt{GetQ()} and \tt{GetR()} return $Q$ and $R$.
\tt{GetP()} returns the permutation array, $P$.
\tt{IsTrans()} returns whether
the decomposition is equal to \tt{m} or \tt{m.Transpose()}.  

The following should result in a \tt{Matrix m2}, which is numerically very close to
the original \tt{Matrix m}:
\begin{tmvcode}
tmv::Matrix<T> m2(m.nrows(),m.ncols());
tmv::MatrixView<T> m2v = 
      m.QRPD().IsTrans() ? m2.Transpose() : m2.View();
m2v = m.QRPD().GetQ() * m.QRPD().GetR();
m2v.ReversePermuteCols(m.QRPD().GetP())
\end{tmvcode}

\item
For the SV decomposition, we have:
\label{matrixsvd}
\begin{tmvcode}
tmv::ConstMatrixView<T> m.SVD().GetU()
tmv::ConstDiagMatrixView<RT> m.SVD().GetS()
tmv::ConstMatrixView<T> m.SVD().GetV()
\end{tmvcode}
\index{Matrix!methods!SVD}
\index{Matrix!singular value decomposition}
\index{Singular Value Decomposition!Matrix}
\tt{GetU()}, \tt{GetS()}, and \tt{GetV()} return $U$, $S$, and $V$.

The following should result in a \tt{Matrix m2}
that is numerically very close to the original \tt{Matrix m}:
\begin{tmvcode}
tmv::Matrix<T> m2 = m.SVD().GetU() * m.SVD().GetS() * m.SVD().GetV();
\end{tmvcode}
\end{itemize}

Each of the above access methods may also be used to perform the decomposition.
You do not need to have used the matrix to perform a division before
calling \tt{m.LUD()}, \tt{m.QRD()}, \tt{m.QRPD()}, or \tt{m.SVD()}.  Also, calling
\tt{m.LUD()} implicitly calls \tt{m.DivideUsing(tmv::LU)} (likewise for the other 
decompositions).  Thus, if you use one of these decomposition methods, any
subsequent division statement will use that decomposition, unless you explicitly
call \tt{DivideUsing} or call a different decomposition method.

Note that a matrix can only store one decomposition at a time.  So if you 
call \tt{m.LUD()} then \tt{m.QRD()}, the LU decomposition is deleted
and the QR decomposition is calculated.  If you call \tt{m.LUD()} again, then 
it will have to be recalculated.

\subsubsection{Singular matrices}
\index{Matrix!singular}
\index{Singular matrices}
\label{singular}

If a matrix is singular (i.e. its determinant is 0), 
then LU and QR decompositions
will fail when you attempt to divide by the matrix, since the 
calculation involves division by 0.  When this happens, the TMV code
throws an exception,
\tt{tmv::Singular}. 
\index{Exceptions!Singular}
Furthermore, with numerical rounding errors, a matrix
that is close to singular may also end up with 0 in a location that gets
used for division and thus throw \tt{tmv::Singular}.

Or, more commonly, a singular or nearly singular matrix
will just have numerically very small values rather than actual 0's.  In this 
case, there won't be an error, but the results will be numerically
very unstable, since the calculation will involve dividing by numbers
that are comparable to the machine precision, $\epsilon$.

You can check whether a Matrix is (exactly) singular with 
the method
\begin{tmvcode}
bool m.Singular()
\end{tmvcode}
\index{Matrix!methods!Singular}
which basically just returns whether \tt{m.Det() == 0}.  But this will not
tell you if a matrix is merely close to singular, so it does not guard 
against unreliable results.

Singular value decompositions provides a way to deal with 
singular and nearly singular matrices.  There are a number of methods for the 
object returned by \tt{m.SVD()}, which can help diagnose and fix 
potential problems with a singular matrix.

First, the so-called ``singular values'', which are the elements of the
diagonal $S$ matrix, tell you how close the matrix is to being singular.
Specifically, if the ratio of the smallest and the largest singular values,
$S(N-1)/S(0)$, is close to the machine precision, $\epsilon$, for the 
underlying type (\tt{double}, \tt{float}, etc.),
then the matrix is singular, or effectively so.
Note: the value of $\epsilon$ is accessible with:
\begin{tmvcode}
std::numeric_limits<T>::epsilon()
\end{tmvcode}

The inverse of this ratio, $S(0)/S(N-1)$, is known as the ``condition'' of the matrix
(specifically the 2-condition, or $\kappa_2$), which can be obtained by:
\begin{tmvcode}
m.SVD().Condition()
\end{tmvcode}
\index{Matrix!methods!SVD}
The larger the condition, the closer the matrix is to singular, and the
less reliable any calculation would be.

So, how does the SVD help in this situation?  (So far we have diagnosed
the possible problem, but not fixed it.)

Well, we need to figure out what solution we want from a singular matrix.
If the matrix is singular, then there are not necessarily any solutions
to $A x = b$.  Furthermore, if we are looking for a least squares solution 
(rather than an exact solution) then there are an infinite number of choices for 
$x$ that give the same minimum value of $||A x-b||_2$.)

Another way of looking at is is that there will be particular values of $y$ 
for which $A y = 0$.
Then given a solution $x$, the vector $x^\prime = x + \alpha y$ for any $\alpha$
will produce 
the same solution: $A x^\prime = A x = b$.

The usual desired solution is the $x$ with minimum 2-norm, $||x||_2$.
With the SVD, we can get this solution by setting to 0 all of the values
in $S^{-1}$ that would otherwise be infinity (or at least large compared
to $1/\epsilon$).  It is somewhat ironic that the best way to deal with 
an infinite value is to set it to 0, but that is actually the solution we want.

There are two methods that can be used to control which 
singular values are set to 0\footnote{Technically, the actual values are preserved,
but an internal value, \tt{kmax}, keeps track of how many singular values to use.}:
\begin{tmvcode}
m.SVD().Thresh(RT thresh)
m.SVD().Top(int nsing)
\end{tmvcode}
\index{Matrix!methods!SVD}
\index{Matrix!singular value decomposition}
\tt{Thresh} sets to 0 any singular values with $S(i)/S(0) <$ \tt{thresh}.
\tt{Top} uses only the largest \tt{nsing} singular values, and sets the rest to 0.

The default behavior is equivalent to:
\begin{tmvcode}
m.SVD().Thresh(std::numeric_limits<T>::epsilon());
\end{tmvcode}
\index{Matrix!methods!SVD}
since at least these values are unreliable.  For different applications,
you may want to use a larger threshold value.

You can check how many singular values are currently considered non-zero with
\begin{tmvcode}
int m.SVD().GetKMax()
\end{tmvcode}
\index{Matrix!methods!SVD}

A QRP decomposition can deal with singular matrices similarly,
but it doesn't have the flexibility in checking for not-quite-singular
but somewhat ill-conditioned matrices like the SVD does.
QRP will put all of the small elements of R's diagonal in the 
lower right corner.  Then it ignores any that are less than 
$\epsilon$ when doing the division.  For actually singular matrices,
this should produce essentially the same result as the SVD solution.
\index{Matrix!QRP decomposition}

We should also mention again that the 2-norm of a matrix is the 
largest singular value, which is just $S(0)$ in our decomposition.
So this norm requires a calculation of the
SVD, which is relatively expensive compared to the other norms
if you have not already calculated the SVD
(and saved it with \tt{m.SaveDiv()}).  On the other hand, if you have already
computed the SVD for division, then \tt{Norm2} is trivial and is the fastest norm to compute.

If you just want to calculate the singular values,
but don't need to do the actual division, then you don't need to 
accumulate the $U$ and $V$ matrices.  This saves a lot of the 
calculation time.  Or you might want $U$ or $V$, but not both for 
some purpose.  See section \S\ref{Decompositions} for how to do this.

\subsection{Input/Output}
\index{Matrix!input/output}

The simplest output is the usual:
\begin{tmvcode}
os << m
\end{tmvcode}
where \tt{os} is any \tt{std::ostream}.
The output format is:
\begin{tmvcode}
nrows ncols
( m(0,0)  m(0,1)  m(0,2)  ...  m(0,ncols-1) )
( m(1,0)  m(1,1)  m(1,2)  ...  m(1,ncols-1) )
...
( m(nrows-1,0) ...  ...  m(nrows-1,ncols-1) )
\end{tmvcode}
\label{matrixio}

The same format can be read back in one of two ways:
\begin{tmvcode}
tmv::Matrix<T> m(nrows,ncols);
is >> m;
std::auto_ptr<tmv::Matrix<T> > pm;
is >> pm;
\end{tmvcode}
For the first version, the \tt{Matrix} must already be declared, which 
requires knowing how big it needs to be.  If the input \tt{Matrix} does not
match in size, a runtime error will occur.
The second version allows you to get the size from the input.  \tt{m2}
will be created (with \tt{new}) 
according to whatever size the input \tt{Matrix} is.

Often, it is convenient to output only those values that aren't very small. 
This can be done using
\begin{tmvcode}
m.Write(std::ostream& os, RT thresh)
\end{tmvcode}
\index{Matrix!methods!Write}
which is equivalent to
\begin{tmvcode}
os << tmv::Matrix<T>(m).Clip(thresh);
\end{tmvcode}
but without requiring the temporary \tt{Matrix}.

\subsection{Small matrices}
\index{SmallVector}
\index{SmallMatrix}
\label{smallmatrix}

The algorithms for regular \tt{Matrix} operations are optimized to be
fast for large matrices.  Usually, this makes sense, since any code with
both large and small matrices will probably have its performance dominated
by the speed of the large matrix algorithms.

However, it may be the case that a particular program spends all of its
time using $2 \times 2$ or $3 \times 3$ matrices.  In this case, 
many of the features of the TMV code are undesirable.  For example, 
the alias checking in the operation \tt{v2 = m * v1} becomes a significant
fraction of the operating time.  Even the simple act of performing a function
call, rather than doing the calculation inline may be a big performance hit.

So we include the alternate matrix class called \tt{SmallMatrix}, along 
with the corresponding vector class called \tt{SmallVector}, for which most of 
the operations are done inline.  Furthermore, the sizes are template arguments,
rather than normal parameters.  This allows the compiler to easily optimize
simple calculations that might only be 2 or 3 arithmetic operations, which
may significantly speed up your code.

All the \tt{SmallMatrix} and \tt{SmallVector} routines are included by:
\begin{tmvcode}
#include "TMV_Small.h"
\end{tmvcode}
\index{TMV\_Small.h}

These classes do not inherit from the regular versions, but they do have 
essentially all the same methods, functions, and arithmetic operators. 

\subsubsection{Constructors}
\index{SmallVector!constructors}
\index{SmallMatrix!constructors}

The template arguments \tt{M} and \tt{N} below are both integers and
represent the size of the matrix or vector as indicated.
The template argument \tt{stor} may be either \tt{tmv::RowMajor} or
\tt{tmv::ColMajor}, and
 \tt{index} may be either \tt{tmv::CStyle} or
\tt{tmv::FortranStyle}.  These both have the same meanings as they do 
for a regular \tt{Matrix}. The defaults are \tt{ColMajor} and \tt{CStyle} if they
are omitted.

\begin{itemize}
\item 
\begin{tmvcode}
tmv::SmallVector<T,N,index> v()
\end{tmvcode}
Makes a \tt{SmallVector} of size \tt{N}
with \underline{uninitialized} values.
If debugging is turned on (i.e. not turned off
with -DNDEBUG), then the values are in fact initialized to 888. 

\item
\begin{tmvcode}
tmv::SmallVector<T,N,index> v(T x)
\end{tmvcode}
Makes a \tt{SmallVector} with all values equal to \tt{x}.

\item
\begin{tmvcode}
tmv::SmallVector<T,N,index> v(const T* vv)
tmv::SmallVector<T,N,index> v(const std::vector<T>& vv)
\end{tmvcode}
Makes a \tt{SmallVector} with values copied from vv.

\item 
\begin{tmvcode}
tmv::SmallVector<T,N,stor,index> v1(const GenVector<T>& v2)
\end{tmvcode}
Makes a \tt{SmallVector} from a regular \tt{Vector}.

\item
\begin{tmvcode}
tmv::SmallVector<T,N,stor,index> v1(
      const SmallVector<T2,N,stor2,index2>& v2)
v1 = v2
\end{tmvcode}
Copy the \tt{SmallVector v2}, which may be of any type \tt{T2} so long
as values of type \tt{T2} are convertible into type \tt{T}.
The assignment operator has the same flexibility.

\item
\tt{v = tmv::ListInit,} \emph{comma-delimited list of values}\vspace{5 pt}\\
Initialize the \tt{SmallVector v} with a list of values.
\index{ListInit}

\item 
\begin{tmvcode}
tmv::SmallMatrix<T,M,N,stor,index> m()
\end{tmvcode}
Makes an \tt{M} $\times$ \tt{N} \tt{SmallMatrix} 
with \underline{uninitialized} values.
If debugging is turned on (i.e. not turned off
with -DNDEBUG), then the values are in fact initialized to 888. 

\item
\begin{tmvcode}
tmv::SmallMatrix<T,M,N,stor,index> m(T x)
\end{tmvcode}
Makes an \tt{M} $\times$ \tt{N} \tt{SmallMatrix} with all values equal to \tt{x}.

\item
\begin{tmvcode}
tmv::SmallMatrix<T,M,N,stor,index> m(const T* vv)
tmv::SmallMatrix<T,M,N,stor,index> m(const std::vector<T>& vv)
tmv::SmallMatrix<T,M,N,stor,index> m(
      const std::vector<std::vector<T> >& vv)
\end{tmvcode}
Makes an \tt{M} $\times$ \tt{N} \tt{SmallMatrix} with values copied from vv.

\item 
\begin{tmvcode}
tmv::SmallMatrix<T,M,N,stor,index> m1(const GenMatrix<T>& m2)
\end{tmvcode}
Makes an \tt{M} $\times$ \tt{N} \tt{SmallMatrix} from a regular \tt{Matrix}.

\item
\begin{tmvcode}
tmv::SmallMatrix<T,M,N,stor,index> m1(
      const SmallMatrix<T2,M,N,stor2,index2>& m2)
m1 = m2
\end{tmvcode}
Copy the \tt{SmallMatrix m2}, which may be of any type \tt{T2} so long
as values of type \tt{T2} are convertible into type \tt{T}.
The assignment operator has the same flexibility.

\item
\tt{m = tmv::ListInit,} \emph{comma-delimited list of values}\vspace{5 pt}\\
Initialize the \tt{SmallMatrix m} with a list of values.  The elements are assigned
in order according to the storage order of \tt{m}.
\index{ListInit}

\end{itemize}


\subsubsection{Access}
\index{SmallVector!access methods}
\index{SmallMatrix!access methods}

The basic access methods are the same as for a regular \tt{Vector} or \tt{Matrix}.
(See \ref{vectoraccess}, \ref{matrixaccess}.)
However, since the size is known to the compiler, the inline calculation is able
to be a lot faster, often reducing to a trivial memory access.

The various view methods, like \tt{row}, \tt{col}, \tt{Transpose}, \tt{Conjugate}, etc. 
do not return a \tt{SmallVector} or \tt{SmallMatrix}, so operations 
with the returned views will not be done inline.
However, you can copy the view back to a ``\tt{Small}'' object, which will be done 
inline, so that should be fast.

\index{SmallVector!views}
\index{SmallMatrix!views}
Also, views may be combined with \tt{Small} objects in arithmetic statements.
In fact, simple views -- those where all the memory 
elements are contiguous in memory -- will usually have the arithmetic calculation
done inline.
In this case, a good optimizing compiler will probably produce code that 
eliminates the temporary
non-\tt{Small} view.  For example:
\begin{tmvcode}
SmallMatrix<T,M,N> m3 = m1.Transpose() * m2;
SmallVector<T,N> v2 = m1 * m3.col(2);
\end{tmvcode}
will both be done inline, even though \tt{m1.Transpose()} is not a \tt{SmallMatrix} and
\tt{m3.col(2)} is not a \tt{SmallVector}.

\subsubsection{Functions}
\index{SmallVector!functions of}
\index{SmallMatrix!functions of}

\tt{SmallVector}s and \tt{SmallMatrix}es all have exactly the same
function methods as the regular varieties.  
(See \ref{vectorfunctions}, \ref{matrixfunctions}.)
Likewise, the syntax of the
arithmetic is identical.  There are only a few methods that 
are not done inline.  

First, reading a \tt{SmallMatrix} or \tt{SmallVector} from a file 
uses the regular
\tt{Matrix} I/O methods.  Also, there is no \tt{auto\_ptr} 
version of these read operations, since you need to know the 
size of a \tt{SmallMatrix} or \tt{SmallVector} at compile time anyway, 
so there is no way to wait until the file is read to determine the size.

Also, the \tt{Sort} command for a \tt{SmallVector} just uses the
regular \tt{Vector} version.

\subsubsection{Limitations}
\label{smallmatrixlimitations}
\index{SmallMatrix!limitations}

There are a few limitations on \tt{SmallMatrix} and \tt{SmallVector} objects
that we impose in order to give the compiler the ability to
optimize them as much as possible.

\begin{enumerate}

\item
\textbf{No alias checking}

\index{SmallVector!arithmetic!no alias checking}
\index{SmallMatrix!arithmetic!no alias checking}
Statements such as 
\begin{tmvcode}
v = m * v;
\end{tmvcode}
will not produce correct code if \tt{v} and \tt{m} are a \tt{SmallVector} and \tt{SmallMatrix}.  

For a regular \tt{Vector} and \tt{Matrix}, the TMV library checks whether any of 
the objects in an arithmetic statement use the same memory.  
Then it uses the correct algorithm to deal with it 
correctly.  

This alias checking is fairly expensive for small matrices, so we don't do it if
\tt{m} and \tt{v} are a \tt{SmallMatrix} and a \tt{SmallVector}.  

If you need to perform this operation you can write:
\begin{tmvcode}
v = m * tmv::SmallVector<N>(v);
\end{tmvcode}
to create a temporary copy of the vector on the right hand side. 

\item
\textbf{No virtual functions}

The vtable required for \tt{SmallMatrix} to use virtual functions is potentially a 
detriment to performance, so we don't have any virtual functions in either \tt{SmallVector}
or \tt{SmallMatrix}. 

A consequence of this decision is that the \tt{SmallMatrix} class is not in the usual
class hierarchy.  It
does not inherit from either \tt{GenMatrix} or \tt{BaseMatrix} or even \tt{AssignableToMatrix}, 
since they have virtual functions.  
Likewise, \tt{SmallVector} does not inherit from \tt{GenVector} or \tt{AssignableToVector}.

If you want to view a \tt{SmallMatrix} as a \tt{GenMatrix}, you can type
\begin{tmvcode}
ConstMatrixView<T> mv = m.View();
\end{tmvcode}
and use mv wherever you need a \tt{GenMatrix}.  Of course, you will lose the 
inlining advantages of a \tt{SmallMatrix}, but sometimes that is what you need.
\index{SmallVector!views}
\index{SmallMatrix!views}

Similarly, a non-\tt{const} \tt{SmallMatrix} can be viewed as a \tt{MatrixView}
with the same method.

\item
\textbf{No implicit instantiation}
\index{SmallVector!arithmetic!no implicit instantiation}
\index{SmallMatrix!arithmetic!no implicit instantiation}

Expressions that involve arithmetic of ``\tt{Small}'' objects do not automatically
instantiate a temporary matrix when necessary.  Therefore, arithmetic 
expressions cannot get as complicated as with regular \tt{Matrix} and 
\tt{Vector} objects.

For example:
\begin{tmvcode}
v3 = m*v2 + 8*v1;
\end{tmvcode}
will not compile.  You need to do this in two steps: first the matrix-vector product, 
and then the addition:
\begin{tmvcode}
v3 = m*v2;
v3 += 8*v1;
\end{tmvcode}

Similarly, you cannot simply output the result of an arithmetic expression without
first assigning it to a result object:
\begin{tmvcode}
std::cout<<"Result is "<<m*v<<std::endl;          // Wrong.
tmv::SmallVector<double,N> result;
std::cout<<"Result is "<<(result=m*v)<<std::endl; // Right.
std::cout<<"Result is "<<tmv::Vector<double>(m*v)<<std::endl; // Right.
\end{tmvcode}

\item
\textbf{Views are not ``\tt{Small}''}
\index{SmallVector!views}
\index{SmallMatrix!views}

All the operations that return some kind of view,
either a \tt{VectorView} or a \tt{MatrixView}, are not ``\tt{Small}''.
As described above, this means that they don't have all the inlining advantages of 
\tt{SmallVector} and \tt{SmallMatrix}.  

However, you can copy them back to a \tt{Small} object or combine
with \tt{Small} objects in an arithmetic statement, which will be done inline
in some cases.

\item
\textbf{Division method is fixed}
\index{SmallMatrix!arithmetic!division}

A \tt{SmallMatrix} does not have the various division control methods 
like \tt{DivideUsing}, \tt{SaveDiv}, etc.
So a square \tt{SmallMatrix} will always use
LU decomposition, and a non-square one will always use QR decomposition.
And if you are doing multiple division statements with the same matrix,
the library will not save the decomposition between statements.  
\index{LU Decomposition!SmallMatrix}
\index{QR Decomposition!SmallMatrix}
\index{SmallMatrix!LU decomposition}
\index{SmallMatrix!QR decomposition}

\end{enumerate}

\newpage
\section{Special Matrices}

Most functions and methods for the various special matrix varieties work the same
as the for regular dense, rectangular matrices.  
In these cases, we will just list the functions
that are allowed for each special matrix with the
effect understood to be the same as for a regular \tt{Matrix}.  Of course, there are usually 
algorithmic speed-ups, which the code will use to take advantage of the 
particular structure.
Whenever there is a difference in how a function works,
we will explain the difference.

\subsection{Diagonal matrices}
\index{DiagMatrix}

The \tt{DiagMatrix} class is our diagonal matrix class.  
A diagonal matrix is only non-zero
along the main diagonal of the matrix.  

The class \tt{DiagMatrix} inherits from \tt{GenDiagMatrix},
which in turn inherits from \tt{BaseMatrix}.
The various views and composite classes described below
also inherit from \tt{GenDiagMatrix}.
\index{DiagMatrix!DiagMatrixView}
\index{DiagMatrix!ConstDiagMatrixView}
\index{DiagMatrix!GenDiagMatrix}
\index{DiagMatrix!DiagMatrixComposite}
\index{BaseMatrix}

\subsubsection{Constructors}
\index{DiagMatrix!constructors}

As usual, the optional \tt{index} template argument specifies which indexing
style to use.

\begin{itemize}
\item 
\begin{tmvcode}
tmv::DiagMatrix<T,index> d(size_t n)
\end{tmvcode}
Makes an \tt{n} $\times$ \tt{n} \tt{DiagMatrix} with \underline{uninitialized} values.
If debugging is turned on (i.e. not turned off
with -DNDEBUG), then the values along the diagonal are in fact initialized to 888. 

\item
\begin{tmvcode}
tmv::DiagMatrix<T,index> d(size_t n, T x)
\end{tmvcode}
Makes an \tt{n} $\times$ \tt{n} \tt{DiagMatrix} with all values along the diagonal equal to \tt{x}.

\item
\begin{tmvcode}
tmv::DiagMatrix<T,index> m(size_t n, const T* vv)
tmv::DiagMatrix<T,index> m(const std::vector<T>& vv)
\end{tmvcode}
Makes a \tt{DiagMatrix} which copies the elements of \tt{vv}.

\item
\begin{tmvcode}
tmv::DiagMatrix<T,index> d(const GenVector<T>& v)
\end{tmvcode}
Makes a \tt{DiagMatrix} with \tt{v} as the diagonal.

\item 
\begin{tmvcode}
tmv::DiagMatrix<T,index> d(const GenMatrix<T>& m)
\end{tmvcode}
Makes a \tt{DiagMatrix} with the diagonal of \tt{m} as the diagonal.

\item
\begin{tmvcode}
tmv::DiagMatrix<T,index> d1(const GenDiagMatrix<T2>& d2)
d1 = d2
\end{tmvcode}
Copy the \tt{DiagMatrix d2}, which may be of any type \tt{T2} so long
as values of type \tt{T2} are convertible into type \tt{T}.
The assignment operator has the same flexibility.

\item
\begin{tmvcode}
tmv::DiagMatrixView<T,index> d = 
      DiagMatrixViewOf(VectorView<T,index> v)
tmv::ConstDiagMatrixView<T,index> d = 
      DiagMatrixViewOf(ConstVectorView<T,index> v)
\end{tmvcode}
\index{Vector!functions of!DiagMatrixViewOf}
\index{Vector!view as a diagonal matrix}
Makes a \tt{DiagMatrixView} whose diagonal is \tt{v}.

\item
\begin{tmvcode}
tmv::DiagMatrixView<T,index> d = 
      tmv::DiagMatrixViewOf(T* vv, size_t n)
tmv::ConstDiagMatrixView<T,index> d = 
      tmv::DiagMatrixViewOf(const T* vv, size_t n)
\end{tmvcode}
\index{DiagMatrix!view of raw memory}
Make a \tt{DiagMatrixView} whose diagonal consists of the actual memory elements \tt{vv}.

\end{itemize}


\subsubsection{Access}
\index{DiagMatrix!access methods}

\begin{tmvcode}
d.nrows() = d.ncols() = d.colsize() = d.rowsize() = d.size()
d(i,j)
d(i) = d(i,i)
\end{tmvcode}
\index{DiagMatrix!methods!nrows}
\index{DiagMatrix!methods!ncols}
\index{DiagMatrix!methods!rowsize}
\index{DiagMatrix!methods!colsize}
\index{DiagMatrix!methods!size}
\index{DiagMatrix!methods!operator()}
For the mutable \tt{d(i,j)} version, 
\tt{i} must equal \tt{j}.
If \tt{d} is not mutable, then \tt{d(i,j)} with \tt{i}$\neq$\tt{j} returns the 
value 0.

\begin{tmvcode}
d.diag()
\end{tmvcode}
\index{DiagMatrix!methods!diag}

\begin{tmvcode}
d.SubDiagMatrix(int i1, int i2, int istep = 1)
\end{tmvcode}
This is equivalent to \tt{DiagMatrixViewOf(d.diag().SubVector(i1,i2,istep))}.
\begin{tmvcode}
d.Transpose() = d.View()
d.Conjugate() = d.Adjoint()
d.Real()
d.Imag()
\end{tmvcode}
\index{DiagMatrix!methods!SubDiagMatrix}
\index{DiagMatrix!methods!Transpose}
\index{DiagMatrix!methods!Conjugate}
\index{DiagMatrix!methods!Adjoint}
\index{DiagMatrix!methods!View}
\index{DiagMatrix!methods!Real}
\index{DiagMatrix!methods!Imag}

\subsubsection{Functions}
\index{DiagMatrix!functions of}

\begin{tmvcode}
RT d.Norm1() = Norm1(d)
RT d.Norm2() = Norm2(d) = d.DoNorm2()
RT d.NormInf() = NormInf(d)
RT d.MaxAbsElement() = MaxAbsElement(d)
\end{tmvcode}
\index{DiagMatrix!functions of!Norm1}
\index{DiagMatrix!functions of!Norm2}
\index{DiagMatrix!functions of!NormInf}
\index{DiagMatrix!functions of!MaxAbsElement}
\index{DiagMatrix!methods!Norm1}
\index{DiagMatrix!methods!Norm2}
\index{DiagMatrix!methods!DoNorm2}
\index{DiagMatrix!methods!NormInf}
\index{DiagMatrix!methods!MaxAbsElement}
(Actually for a diagonal matrix, all of the above norms are equal.)
\begin{tmvcode}
RT d.NormF() = NormF(d) = d.Norm() = Norm(d)
RT d.NormSq() = NormSq(d)
RT d.NormSq(RT scale)
T d.Trace() = Trace(d)
T d.Det() = Det(d)
RT d.LogDet(T* sign=0) = LogDet(d)
bool d.Singular()
RT d.Condition()
RT d.DoCondition()
dinv = d.Inverse() = Inverse(d)
d.Inverse(Matrix<T>& minv)
d.Inverse(DiagMatrix<T>& dinv)
d.InverseATA(Matrix<T>& cov)
d.InverseATA(DiagMatrix<T>& cov)
\end{tmvcode}
\index{DiagMatrix!functions of!Norm}
\index{DiagMatrix!functions of!NormF}
\index{DiagMatrix!functions of!NormSq}
\index{DiagMatrix!functions of!Trace}
\index{DiagMatrix!functions of!Det}
\index{DiagMatrix!functions of!LogDet}
\index{DiagMatrix!functions of!Inverse}
\index{DiagMatrix!methods!Norm}
\index{DiagMatrix!methods!NormF}
\index{DiagMatrix!methods!NormSq}
\index{DiagMatrix!methods!Trace}
\index{DiagMatrix!methods!Det}
\index{DiagMatrix!methods!LogDet}
\index{DiagMatrix!methods!Singular}
\index{DiagMatrix!methods!Condition}
\index{DiagMatrix!methods!DoCondition}
\index{DiagMatrix!methods!Inverse}
\index{DiagMatrix!methods!InverseATA}
Since the inverse of a \tt{DiagMatrix} is a \tt{DiagMatrix},
we also provide a version of the \tt{Inverse} syntax, which allows dinv
to be a \tt{DiagMatrix}.  (Likewise for \tt{InverseATA}.)  The same option is 
available with the operator version: \tt{dinv = d.Inverse()}.

\begin{tmvcode}
d.Zero()
d.SetAllTo(T x)
d.Clip(RT thresh)
d.SetToIdentity(T x = 1)
d.ConjugateSelf()
d.TransposeSelf() // null operation
d.InvertSelf()
Swap(d1,d2)
\end{tmvcode}
\index{DiagMatrix!methods!Zero}
\index{DiagMatrix!methods!SetAllTo}
\index{DiagMatrix!methods!Clip}
\index{DiagMatrix!methods!SetToIdentity}
\index{DiagMatrix!methods!ConjugateSelf}
\index{DiagMatrix!methods!TransposeSelf}
\index{DiagMatrix!methods!InvertSelf}
\index{DiagMatrix!functions of!Swap}
\tt{InvertSelf} is new for \tt{DiagMatrix} and calculates $d^{-1}$ in place.  
It is equivalent to \tt{d = d.Inverse()}.

\subsubsection{Arithmetic}
\index{DiagMatrix!arithmetic}

In addition to \tt{x}, \tt{v}, and \tt{m} from before, we now add \tt{d} for a \tt{DiagMatrix}.

\begin{tmvcode}
d2 = -d1
d2 = x * d1
d2 = d1 [*/] x
d3 = d1 [+-] d2
m2 = m1 [+-] d
m2 = d [+-] m1
d [*/]= x
d2 [+-]= d1
m [+-]= d
v2 = d * v1
v2 = v1 * d
v *= d
d3 = d1 * d2
m2 = d * m1
m2 = m1 * d
d2 *= d1
m *= d
d2 = d1 [+-] x
d2 = x [+-] d1
d [+-]= x
d = x
\end{tmvcode}

\subsubsection{Division}
\index{DiagMatrix!arithmetic!division}

The division operations are:
\begin{tmvcode}
v2 = v1 [/%] d
m2 = m1 [/%] d
m2 = d [/%] m1
d3 = d1 [/%] d2
d2 = x [/%] d1
v [/%]= d
d2 [/%]= d1
m [/%]= d
\end{tmvcode}

There is only one allowed \tt{DivType} for a \tt{DiagMatrix}: \tt{LU}. 
And, since it is also the default behavior,
there is no reason to ever use this function.  Furthermore, since a \tt{DiagMatrix}
is already a $U$ matrix, the decomposition requires no work at all.
Hence, it is always done in place; no extra storage is needed, and 
the methods \tt{m.DivideInPlace()}, \tt{m.SaveDiv()}, etc. are irrelevant.

If a \tt{DiagMatrix} is singular, you can find out with \tt{m.Singular()},
but there is no direct way to use SVD for the division and skip any
divisions by 0.  If you want to do this, you should use \tt{BandMatrixViewOf(d)} to 
treat the \tt{DiagMatrix} as a \tt{BandMatrix}, which can use SVD.

\subsubsection{Input/Output}
\index{DiagMatrix!input/output}

The simplest output is the usual:
\begin{tmvcode}
os << d
\end{tmvcode}
where \tt{os} is any \tt{std::ostream}.
The output format is the same as for a \tt{Matrix}, including all the 0's.
(See \ref{matrixio}.)


There is also a compact format:
\begin{tmvcode}
d.WriteCompact(os)
\end{tmvcode}
\index{DiagMatrix!methods!WriteCompact}
which outputs in the format:
\begin{tmvcode}
D n ( d(0,0)  d(1,1)  d(2,2)  ...  d(n-1,n-1) )
\end{tmvcode}

The same (compact, that is) format can be read back in the usual two ways:
\begin{tmvcode}
tmv::DiagMatrix<T> d(n);
is >> d;
std::auto_ptr<tmv::DiagMatrix<T> > pd;
is >> pd;
\end{tmvcode}
where the first gives an error if \tt{d} is the wrong size and the second allocates
a new \tt{DiagMatrix} that is the correct size.

One can also write small values as 0 with
\begin{tmvcode}
m.Write(std::ostream& os, RT thresh)
m.WriteCompact(std::ostream& os, RT thresh)
\end{tmvcode}
\index{DiagMatrix!methods!Write}
\index{DiagMatrix!methods!WriteCompact}

\subsection{Upper/lower triangle matrices}
\index{UpperTriMatrix}
\index{LowerTriMatrix|see{UpperTriMatrix}}
\label{trimatrix}

The \tt{UpperTriMatrix} class is our upper triangle matrix class, which is non-zero
only on the main diagonal and above.  \tt{LowerTriMatrix} is our class for lower
triangle matrices, which are non-zero only on the main diagonal and below.

The class \tt{UpperTriMatrix} inherits from \tt{GenUpperTriMatrix}, 
and the class \tt{LowerTriMatrix}
inherits from \tt{GenLowerTriMatrix}, both of
which in turn inherit from \tt{BaseMatrix}.  
The various views and composite classes described below
also inherit from \tt{GenUpperTriMatrix} and \tt{GenLowerTriMatrix} 
as appropriate.
\index{UpperTriMatrix!UpperTriMatrixView}
\index{UpperTriMatrix!ConstUpperTriMatrixView}
\index{UpperTriMatrix!GenUpperTriMatrix}
\index{UpperTriMatrix!UpperTriMatrixComposite}
\index{BaseMatrix}

In addition to the \tt{T} template parameter, there are three other template 
parameters:
\tt{dt}, which can be either \tt{tmv::UnitDiag} or \tt{tmv::NonUnitDiag};
\tt{stor}, which can be \tt{tmv::RowMajor} or \tt{tmv::ColMajor};
and \tt{index}, which can be \tt{tmv::CStyle} or \tt{tmv::FortranStyle}.
The default values for these template parameters are 
\tt{NonUnitDiag}, \tt{ColMajor}, and \tt{CStyle} respectively.

The storage of both an \tt{UpperTriMatrix} and a \tt{LowerTriMatrix} takes
$N \times N$ elements of memory, even though approximately half of them 
are never used.  Someday, I'll write the packed storage versions, which allow for
efficient storage of the matrices.

All of the routines are analogous for \tt{UpperTriMatrix} and 
\tt{LowerTriMatrix}, so we only list each routine once 
(the \tt{UpperTriMatrix} version for
definiteness).  

\subsubsection{Constructors}
\index{UpperTriMatrix!constructors}

\begin{itemize}
\item 
\begin{tmvcode}
tmv::UpperTriMatrix<T,dt,stor,index> U(size_t n)
\end{tmvcode}
Makes an \tt{n} $\times$ \tt{n} \tt{UpperTriMatrix} with \underline{uninitialized} values.
If debugging is turned on (i.e. not turned off
with -DNDEBUG), then the values are in fact initialized to 888.  

\item
\begin{tmvcode}
tmv::UpperTriMatrix<T,dt,stor,index> U(size_t n, T x)
\end{tmvcode}
Makes an \tt{n} $\times$ \tt{n} \tt{UpperTriMatrix} with all values equal to \tt{x}.

\item
\begin{tmvcode}
tmv::UpperTriMatrix<T,dt,stor,index> U(size_t n, const T* vv)
tmv::UpperTriMatrix<T,dt,stor,index> U(size_t n, 
      const std::vector<T>& vv)
\end{tmvcode}
Makes an \tt{UpperTriMatrix} which copies the elements of \tt{vv}.
Note: the elements in \tt{vv} need to include the zero's of the lower triangle 
portion of \tt{U}.

\item 
\begin{tmvcode}
tmv::UpperTriMatrix<T,dt,stor,index> U(const GenMatrix<T>& m)
tmv::UpperTriMatrix<T,dt,stor,index> U(const GenUperTriMatrix<T>& U2)
\end{tmvcode}
Make an \tt{UpperTriMatrix} which copies the corresponding values of
\tt{U2}.  Note that the second one is allowed to have \tt{U2} be
\tt{NonUnitDiag} but \tt{dt = UnitDiag}, in which case only the 
off-diagonal elements are copied.  The converse would set the 
diagonal of the new \tt{UpperTriMatrix} to all 1's.

\item
\begin{tmvcode}
tmv::UpperTriMatrix<T> U1(const GenUpperTriMatrix<T2>& U2)
U1 = U2
\end{tmvcode}
Copy the \tt{UpperTriMatrix U2}, which may be of any type \tt{T2} so long
as values of type \tt{T2} are convertible into type \tt{T}.
The assignment operator has the same flexibility.

\item
\begin{tmvcode}
tmv::UpperTriMatrixView<T,index> U = 
      tmv::UpperTriMatrixViewOf(T* vv, size_t n, 
      StorageType stor, DiagType dt=NonUnitDiag)
tmv::ConstUpperTriMatrixView<T,index> U = 
      tmv::UpperTriMatrixViewOf(const T* vv, size_t n, 
      StorageType stor, DiagType dt=NonUnitDiag)
\end{tmvcode}
\index{UpperTriMatrix!view of raw memory}
Make a \tt{UpperTriMatrixView} of the actual memory elements, \tt{vv}.
One wrinkle here is that if \tt{dt} is \tt{UnitDiag}, then 
\tt{vv} is still the location of the
upper left corner, even though that value is never used 
(since the value is just taken to
be 1).  Also, \tt{vv} must be of length \tt{n} $\times$ \tt{n},
so all of the lower triangle
elements must be in memory, even though they are never used.

\end{itemize}

\subsubsection{Access}
\index{UpperTriMatrix!access methods}

\begin{tmvcode}
U.nrows() = U.ncols() = U.colsize() = U.rowsize() = U.size()
U(i,j)
U.row(int i, int j1, int j2)
U.col(int i, int j1, int j2)
U.diag()
U.diag(int i)
U.diag(int i, int k1, int k2)
\end{tmvcode}
\index{UpperTriMatrix!methods!nrows}
\index{UpperTriMatrix!methods!ncols}
\index{UpperTriMatrix!methods!rowsize}
\index{UpperTriMatrix!methods!colsize}
\index{UpperTriMatrix!methods!size}
\index{UpperTriMatrix!methods!operator()}
\index{UpperTriMatrix!methods!row}
\index{UpperTriMatrix!methods!col}
\index{UpperTriMatrix!methods!diag}
For the mutable \tt{d(i,j)} version, 
Note that the versions of \tt{row} and \tt{col} with only one argument are
missing, since the full row or column isn't accessible as a \tt{VectorView}.
You must specify a valid range within the row or column that you want, 
given the upper triangle shape of \tt{U}.  Likewise for the \tt{LowerTriMatrix}
versions of these.
If \tt{dt} is \tt{UnitDiag}, then the range may not include the diagonal element.
Similarly, \tt{U.diag()} is valid only if \tt{dt} is \tt{NonUnitDiag}.

\begin{tmvcode}
U.SubVector(int i, int j, int istep, int jstep, int size)
U.SubMatrix(int i1, int i2, int j1, int j2)
U.SubMatrix(int i1, int i2, int j1, int j2, int istep, int jstep)
\end{tmvcode}
\index{UpperTriMatrix!methods!SubVector}
\index{UpperTriMatrix!methods!SubMatrix}
This works the same as for \tt{Matrix}
(See \ref{matrixaccess}),
except that all of the elements in the 
subvector or submatrix must be completely within the upper or lower triangle, as
appropriate.  If \tt{dt} is \tt{UnitDiag}, then no elements may be on the 
main diagonal.

\begin{tmvcode}
U.SubTriMatrix(int i1, int i2, int istep = 1)
\end{tmvcode}
\index{UpperTriMatrix!methods!SubTriMatrix}
This returns the upper or lower triangle matrix whose upper-left
corner is \tt{U(i1,i1)}, and whose lower-right corner is 
\tt{U(i2-istep,i2-istep)} for C-style indexing or \tt{U(i2,i2)} 
for Fortran-style indexing.  If \tt{istep} $\neq 1$, then it is the 
step in both the \tt{i} and \tt{j} directions.

\begin{tmvcode}
U.OffDiag()
\end{tmvcode}
\index{UpperTriMatrix!methods!OffDiag}
This returns a view to the portion of the triangle matrix that does not include
the diagonal elements.  It will always be \tt{NonUnitDiag}.
Internally, it provides an easy way to deal with the \tt{UnitDiag} triangle matrices
for many routines.  But it may be useful for some users as well.

\begin{tmvcode}
U.ViewAsUnitDiag()
\end{tmvcode}
\index{UpperTriMatrix!methods!ViewAsUnitDiag}
This returns a view to a \tt{NonUnitDiag} triangle matrix that treats it
instead as a \tt{UnitDiag} triangle matrix.

\begin{tmvcode}
U.Transpose()
U.Conjugate()
U.Adjoint()
U.View()
U.Real()
U.Imag()
\end{tmvcode}
\index{UpperTriMatrix!methods!Transpose}
\index{UpperTriMatrix!methods!Conjugate}
\index{UpperTriMatrix!methods!Adjoint}
\index{UpperTriMatrix!methods!View}
\index{UpperTriMatrix!methods!Real}
\index{UpperTriMatrix!methods!Imag}
Note that the transpose and adjoint of an \tt{UpperTriMatrix} is an
\tt{LowerTriMatrixView} and vice versa.
\vspace{12pt}

\subsubsection{Functions}
\index{UpperTriMatrix!functions of}

\begin{tmvcode}
RT U.Norm1() = Norm1(U)
RT U.Norm2() = Norm2(U) = U.DoNorm2()
RT U.NormInf() = NormInf(U)
RT U.NormF() = NormF(U) = U.Norm() = Norm(U)
RT U.NormSq() = NormSq(U)
RT U.NormSq(RT scale)
RT U.MaxAbsElement() = MaxAbsElement(U)
T U.Trace() = Trace(U)
T U.Det() = Det(U)
RT U.LogDet(T* sign=0) = LogDet(U)
bool U.Singular()
RT U.Condition()
RT U.DoCondition()
Uinv = U.Inverse() = Inverse(U)
U.Inverse(Matrix<T>& minv)
U.Inverse(UpperTriMatrix<T>& Uinv)
U.InverseATA(Matrix<T>& cov)
\end{tmvcode}
\index{UpperTriMatrix!functions of!Norm1}
\index{UpperTriMatrix!functions of!Norm2}
\index{UpperTriMatrix!functions of!NormInf}
\index{UpperTriMatrix!functions of!MaxAbsElement}
\index{UpperTriMatrix!methods!Norm1}
\index{UpperTriMatrix!methods!Norm2}
\index{UpperTriMatrix!methods!DoNorm2}
\index{UpperTriMatrix!methods!NormInf}
\index{UpperTriMatrix!methods!MaxAbsElement}
\index{UpperTriMatrix!functions of!Norm}
\index{UpperTriMatrix!functions of!NormF}
\index{UpperTriMatrix!functions of!NormSq}
\index{UpperTriMatrix!functions of!Trace}
\index{UpperTriMatrix!functions of!Det}
\index{UpperTriMatrix!functions of!LogDet}
\index{UpperTriMatrix!functions of!Inverse}
\index{UpperTriMatrix!methods!Norm}
\index{UpperTriMatrix!methods!NormF}
\index{UpperTriMatrix!methods!NormSq}
\index{UpperTriMatrix!methods!Trace}
\index{UpperTriMatrix!methods!Det}
\index{UpperTriMatrix!methods!LogDet}
\index{UpperTriMatrix!methods!Singular}
\index{UpperTriMatrix!methods!Condition}
\index{UpperTriMatrix!methods!DoCondition}
\index{UpperTriMatrix!methods!Inverse}
\index{UpperTriMatrix!methods!InverseATA}
Since the inverse of an \tt{UpperTriMatrix} is also upper triangular,
the object returned by \tt{U.Inverse()} is 
assignable to an \tt{UpperTriMatrix}.  Of course you can also assign it
to a regular \tt{Matrix} if you prefer.  Similarly, there are versions
of \tt{U.Inverse(minv)} for both argument types.  Of course, similar statements
hold for \tt{LowerTriMatrix} as well.

\begin{tmvcode}
U.Zero()
U.SetAllTo(T x)
U.Clip(RT thresh)
U.SetToIdentity(T x = 1)
U.ConjugateSelf()
U.InvertSelf()
Swap(U1,U2)
\end{tmvcode}
\index{UpperTriMatrix!methods!Zero}
\index{UpperTriMatrix!methods!SetAllTo}
\index{UpperTriMatrix!methods!Clip}
\index{UpperTriMatrix!methods!SetToIdentity}
\index{UpperTriMatrix!methods!ConjugateSelf}
\index{UpperTriMatrix!methods!InvertSelf}
\index{UpperTriMatrix!functions of!Swap}
Like for \tt{DiagMatrix}, \tt{InvertSelf} calculates $U^{-1}$ in place.  
It is equivalent to \tt{U = U.Inverse()}.
\vspace{12pt}

\subsubsection{Arithmetic}
\index{UpperTriMatrix!arithmetic}

In addition to \tt{x}, \tt{v}, and \tt{m} from before, 
we now add \tt{U} and \tt{L} for a \tt{UpperTriMatrix}
and \tt{LowerTriMatrix} respectively.  Where the syntax is identical
for the two cases, only the \tt{U} form is listed.

\begin{tmvcode}
U2 = -U1
U2 = x * U1
U2 = U1 [*/] x
U3 = U1 [+-] U2
m2 = m1 [+-] U
m2 = U [+-] m1
m = L [+-] U
m = U [+-] L
U [*/]= x
U2 [+-]= U1
m [+-]= U
v2 = U * v1
v2 = v1 * U
v *= U
U3 = U1 * U2
m2 = U * m1
m2 = m1 * U
m = U * L
m = L * U
U2 *= U1
m *= U
U2 = U1 [+-] x
U2 = x [+-] U1
U [+-]= x
\end{tmvcode}

\subsubsection{Division}
\index{UpperTriMatrix!arithmetic!division}

The division operations are: (again omitting the L forms when redundant)
\begin{tmvcode}
v2 = v1 [/%] U
m2 = m1 [/%] U
m2 = U [/%] m1
U3 = U1 [/%] U2
U2 = x [/%] U1
m = U [/%] L
m = L [/%] U
v [/%]= U
U2 [/%]= U1
m [/%]= U
\end{tmvcode}

There is only one allowed \tt{DivType} for an \tt{UpperTriMatrix} or a 
\tt{LowerTriMatrix}: \tt{LU}.
And, since it is also the default behavior,
there is no reason to ever specify it.
Furthermore, as with a \tt{DiagMatrix},
the decomposition requires no work at all.  In fact, the ease of dividing by a 
upper or lower triangle matrix is precisely why the LU decomposition is useful.
Hence, it is always done in place.  i.e. no extra storage is needed, and all of
the \tt{m.DivideInPlace()}, \tt{m.SaveDiv()}, etc. are irrelevant.

If an \tt{UpperTriMatrix} or \tt{LowerTriMatrix} is singular, 
you can check easily with \tt{m.Singular()},
but there is no direct way to use SVD for the division and avoid any
divisions by 0.  If you want to do this use \tt{BandMatrixViewOf(m)} to 
treat the \tt{TriMatrix} as a \tt{BandMatrix}, which can use SVD.

\subsubsection{Input/Output}
\index{UpperTriMatrix!input/output}

The simplest output is the usual:
\begin{tmvcode}
os << U << L
\end{tmvcode}
where \tt{os} is any \tt{std::ostream}.
The output format is the same as for a \tt{Matrix}, including all the 0's.
(See \ref{matrixio}.)

There is also a compact format.  For an \tt{UpperTriMatrix},
\begin{tmvcode}
U.WriteCompact(os)
\end{tmvcode}
\index{UpperTriMatrix!methods!WriteCompact}
outputs in the format:
\begin{tmvcode}
U n 
( U(0,0)  U(0,1)  U(0,2)  ...  U(0,n-1) )
( U(1,1)  U(1,2)  ...  U(1,n-1) )
...
( U(n-1,n-1) )
\end{tmvcode}
For a \tt{LowerTriMatrix},
\begin{tmvcode}
L.WriteCompact(os)
\end{tmvcode}
outputs in the format:
\begin{tmvcode}
L n 
( L(0,0) )
( L(1,0)  L(1,1) )
...
( L(n-1,0)  L(n-1,1) ... L(n-1,n-1) )
\end{tmvcode}

In each case, the compact format can be read back in the usual two ways:
\begin{tmvcode}
tmv::UpperTriMatrix<T> U(n);
tmv::LowerTriMatrix<T> L(n);
is >> U >> L;
std::auto_ptr<tmv::UpperTriMatrix<T> > pU;
std::auto_ptr<tmv::LowerTriMatrix<T> > pL;
is >> pU >> pL;
\end{tmvcode}

One can write small values as 0 with
\begin{tmvcode}
m.Write(std::ostream& os, RT thresh)
m.WriteCompact(std::ostream& os, RT thresh)
\end{tmvcode}
\index{UpperTriMatrix!methods!Write}
\index{UpperTriMatrix!methods!WriteCompact}

\subsection{Band matrices}
\index{BandMatrix}

The \tt{BandMatrix} class is our band-diagonal matrix, which is only non-zero
on the main diagonal and a few sub- and super-diagonals.  While band-diagonal
matrices are usually square, we allow for non-square banded matrices as well.
You may even have rows or columns that are completely outside of the 
band structure, and hence are all 0.  For example a $10\times 5$ band matrix
with 2 sub-diagonals is valid even though the bottom 3 rows are all 0.

Throughout, we use \tt{nlo} to refer to the number of sub-diagonals 
(below the main diagonal) stored
in the \tt{BandMatrix}, and \tt{nhi} to refer to the number of super-diagonals
(above the main diagonal).

\tt{BandMatrix} inherits from \tt{GenBandMatrix}, which in turn inherits from
\tt{BaseMatrix}.  
The various views and composite classes described below 
also inherit from \tt{GenBandMatrix}.
\index{BandMatrix!BandMatrixView}
\index{BandMatrix!ConstBandMatrixView}
\index{BandMatrix!GenBandMatrix}
\index{BandMatrix!BandMatrixComposite}
\index{BaseMatrix}

All the \tt{BandMatrix} routines are included by:
\begin{tmvcode}
#include "TMV_Band.h"
\end{tmvcode}
\index{TMV\_Band.h}

In addition to the \tt{T} template parameter, we also have \tt{stor} to indicate 
which storage you want to use, and the usual \tt{index} parameter.
For this class, we add an additional storage possibility:
along with \tt{RowMajor} and 
\tt{ColMajor} storage, a \tt{BandMatrix} may also be \tt{DiagMajor}, which 
has unit step along the diagonals.  The \tt{index} parameter has the usual
options of \tt{CStyle} or \tt{FortranStyle}.
The default values for \tt{stor} and \tt{index} are \tt{ColMajor} and \tt{CStyle}.

For each type of storage, we require that the step size in each direction
be uniform within a given row, column or diagonal. 
This means that we require a few extra elements
of memory that are not actually used.
To demonstrate the different storage orders and why extra memory is required, 
here are three $6 \times 6$ band-diagonal
matrices, each with $nlo = 2$ and $nhi = 3$ in each of the different storage
types.  The number in each place indicates the offset in memory from the 
top left element.

\begin{align*}
\textrm{ColMajor:} ~ ~ & \left(\begin{array}{cccccc}0 & 5 & 10 & 15 &  &  \\1 & 6 & 11 & 16 & 21 &  \\2 & 7 & 12 & 17 & 22 & 27 \\ & 8 & 13 & 18 & 23 & 28 \\ &  & 14 & 19 & 24 & 29 \\ &  &  & 20 & 25 & 30\end{array}\right) \\
\textrm{RowMajor:} ~  ~ & \left(\begin{array}{cccccc}0 & 1 & 2 & 3 &  &  \\ 5 & 6 & 7 & 8 & 9 &  \\ 10 & 11 & 12 & 13 & 14 & 15 \\ & 16 & 17 & 18 & 19 & 20 \\ &  & 22 & 23 & 24 & 25 \\ &  &  & 28 & 29 & 30\end{array}\right) \\
\textrm{DiagMajor:} ~  ~ & \left(\begin{array}{cccccc}0 & 6 & 12 & 18 &  &  \\-5 & 1 & 7 & 13 & 19 &  \\ -10 & -4 & 2 & 8 & 14 & 20 \\ & -9 & -3 & 3 & 9 & 15 \\ &  & -8 & -2 & 4 & 10 \\ &  &  & -7 & -1 & 5\end{array}\right)
\end{align*}

First, notice that all three storage methods require 4 extra locations in memory, 
which do not hold
any actual matrix data.  
(They require a total of 31 memory addresses for only 27 that are used.)
This is because we want to have the same step size between consecutive row elements
for every row.  Likewise for the columns (which in turn implies that it is also 
true for the diagonals).

For $N\times N$ square matrices, the total memory needed is $(N-1)*(nlo+nhi+1)+1$, 
which wastes $(nlo-1)*nlo/2 + (nhi-1)*nhi/2$ locations.  For non-square
matrices, the formula is more complicated, and changes slightly between the 
three storages.
If you want to know the memory used by a \tt{BandMatrix}, we provide the routine:
\begin{tmvcode}
size_t BandStorageLength(StorageType stor, size_t nrows, size_t ncols, 
        int nlo, int nhi)
\end{tmvcode}
\index{BandMatrix!storage length}
For the mutable \tt{d(i,j)} version, 
\label{bandmatrixstoragesize}
  
For square matrices, all three methods always need the same amount of memory 
(and for non-square, they aren't very different), so the 
decision about which method to use should generally be based on performance 
considerations rather than memory usage.
The speed of the various matrix operations are different for the different storage 
types.  If the matrix calculation speed is important, it may be worth trying 
all three to see which is fastest for the operations you are using.

Second, notice that the \tt{DiagMajor} storage doesn't start with the 
upper left element as usual.
Rather, it starts at the start of the lowest sub-diagonal.  
So for the constructors that 
take the matrix information from an array (\tt{T*} or \tt{vector<T>}), 
the start of the array needs to be at the start of the lowest sub-diagonal.  

\subsubsection{Constructors}
\index{BandMatrix!constructors}

\begin{itemize}
\item 
\begin{tmvcode}
tmv::BandMatrix<T,stor,index> b(size_t nrows, size_t ncols, 
      int nlo, int nhi)
\end{tmvcode}
Makes a \tt{BandMatrix} with \tt{nrows} rows, \tt{ncols} columns, 
\tt{nlo} sub-diagonals,
and \tt{nhi} super-diagonals with \underline{uninitialized} values.
If debugging is turned on (i.e. not turned off 
with -DNDEBUG), then the values are initialized to 888.

\item
\begin{tmvcode}
tmv::BandMatrix<T,stor,index> b(size_t nrows, size_t ncols, 
      int nlo, int nhi, T x)
\end{tmvcode}
Makes a \tt{BandMatrix} with all values equal to \tt{x}.

\item
\begin{tmvcode}
tmv::BandMatrix<T,stor,index> b(size_t nrows, size_t ncols, 
      int nlo, int nhi, const T* vv)
tmv::BandMatrix<T,stor,index> b(size_t nrows, size_t ncols, 
      int nlo, int nhi, const std::vector<T>& vv)
\end{tmvcode}
Makes a \tt{BandMatrix} which copies the elements from \tt{vv}.  See the 
discussion above about the different storage types to see what order these elements
should be.  The function \tt{BandStorageLength} will tell you how
long \tt{vv} must be.  The elements that don't fall in the bounds of the 
actual matrix are not used and may be left undefined.

\item 
\begin{tmvcode}
tmv::BandMatrix<T,stor,index> b(const GenMatrix<T>& m, 
      int nlo, int nhi)
tmv::BandMatrix<T,stor,index> b(const GenBandMatrix<T>& m, 
      int nlo, int nhi)
tmv::BandMatrix<T,stor,index> b(const GenUpperTriMatrix<T>& m,
      int nhi)
tmv::BandMatrix<T,stor,index> b(const GenLowerTriMatrix<T>& m,
      int nlo)
\end{tmvcode}
Make a \tt{BandMatrix} the same size as \tt{m}, which copies the values of \tt{m}
that are within the band defined by \tt{nlo} and \tt{nhi}
For the second one, \tt{nlo} and \tt{nhi} must not be larger than those for \tt{m}.
For the last two, \tt{nlo} and \tt{nhi} (respectively) are taken to be $0$.

\item
\begin{tmvcode}
tmv::BandMatrix<T,tmv::DiagMajor> m = UpperBiDiagMatrix(
      const GenVector<T>& v1, const GenVector<T>& v2)
tmv::BandMatrix<T,tmv::DiagMajor> m = LowerBiDiagMatrix(
      const GenVector<T>& v1, const GenVector<T>& v2)
tmv::BandMatrix<T,tmv::DiagMajor> m = TriDiagMatrix(
      const GenVector<T>& v1,  const GenVector<T>& v2, 
      const GenVector<T>& v3)
\end{tmvcode}
\index{BandMatrix!constructors!UpperBiDiagMatrix}
\index{BandMatrix!constructors!LowerBiDiagMatrix}
\index{BandMatrix!constructors!TriDiagMatrix}
Shorthand to create bi- or tri-diagonal \tt{BandMatrix}es if you already have the 
\tt{Vector}s.  The \tt{Vector}s are in order from bottom to top in each case.

\item
\begin{tmvcode}
tmv::BandMatrix<T,stor,index> b1(const GenBandMatrix<T2>& b2)
b1 = b2
\end{tmvcode}
Copy the \tt{BandMatrix b2}, which may be of any type \tt{T2} so long
as values of type \tt{T2} are convertible into type \tt{T}.
The assignment operator has the same flexibility.

\item
\begin{tmvcode}
tmv::BandMatrixView<T,index> b = 
      BandMatrixViewOf(MatrixView<T,index> m, int nlo, int nhi)
tmv::BandMatrixView<T,index> b = 
      BandMatrixViewOf(BandMatrixView<T,index> m, int nlo, int nhi)
tmv::BandMatrixView<T,index> b = 
      BandMatrixViewOf(DiagMatrixView<T,index> m)
tmv::BandMatrixView<T,index> b = 
      BandMatrixViewOf(UpperTriMatrixView<T,index> m)
tmv::BandMatrixView<T,index> b = 
      BandMatrixViewOf(UpperTriMatrixView<T,index> m, int nhi)
tmv::BandMatrixView<T,index> b = 
      BandMatrixViewOf(LowerTriMatrixView<T,index> m)
tmv::BandMatrixView<T,index> b = 
      BandMatrixViewOf(LowerTriMatrixView<T,index> m, int nlo)
\end{tmvcode}
\index{Matrix!view portion as a BandMatrix}
\index{BandMatrix!view portion as a thinner BandMatrix}
\index{DiagMatrix!view as a BandMatrix}
\index{UpperTriMatrix!view as a BandMatrix}
Make an \tt{BandMatrixView} of the corresponding portion of \tt{m}.  
There are also \tt{ConstBandMatrixView} versions of all of these.  

\item
\begin{tmvcode}
tmv::BandMatrixView<T,index> b(MatrixView<T,index> m, 
      int nlo, int nhi)
tmv::BandMatrixView<T,index> b(BandMatrixView<T,index> m, 
      int nlo, int nhi)
\end{tmvcode}
\index{BandMatrix!constructors!BandMatrixView}
\index{Matrix!view portion as a BandMatrix}
\index{BandMatrix!view portion as a thinner BandMatrix}
For square matrices \tt{m}, these (and the corresponding \tt{ConstBandMatrixView}
versions) work the same as the above \tt{BandMatrixViewOf}
commands.  However, this version preserves the values of \tt{nrows} and \tt{ncols}
from \tt{m}
even if some of the rows or columns do not include any of the new band.
This is only important if \tt{m} is not square.

For example, if \tt{m} is $10 \times 8$, then 
\begin{tmvcode}
tmv::BandMatrixView<T> b1(m,0,2);
\end{tmvcode}
will create a $10 \times 8$ \tt{BandMatrixView} of \tt{m}'s diagonal plus two super-diagonals, but
\begin{tmvcode}
tmv::BandMatrixView<T> b2 = BandMatrixViewOf(m,0,2);
\end{tmvcode}
will instead create an $8 \times 8$ \tt{BandMatrixView} of the same portion of \tt{m}.

Note that the same difference holds for the \tt{BandMatrix} constructor:
\begin{tmvcode}
tmv::BandMatrix<T> b1(m,0,2);
\end{tmvcode}
will create a $10 \times 8$ \tt{BandMatrix}, but
\begin{tmvcode}
tmv::BandMatrix<T> b2 = BandMatrixViewOf(m,0,2);
\end{tmvcode}
will create an $8 \times 8$ \tt{BandMatrix}.

\item
\begin{tmvcode}
tmv::BandMatrixView<T> b = 
      tmv::BandMatrixViewOf(T* vv, size_t ncols, size_t nrows, 
          int nlo, int nhi, StorageType stor)
tmv::ConstBandMatrixView<T> b = 
      tmv::BandMatrixViewOf(const T* vv, size_t ncols, size_t nrows, 
          int nlo, int nhi, StorageType stor)
\end{tmvcode}
\index{BandMatrix!view of raw memory}
Make a \tt{BandMatrixView} of the actual memory elements, \tt{vv}.

\end{itemize}

\subsubsection{Access}
\index{BandMatrix!access methods}

\begin{tmvcode}
b.nrows() = b.colsize()
b.ncols() = b.rowsize()
b(i,j)
b.row(int i, int j1, int j2)
b.col(int i, int j1, int j2)
b.diag()
b.diag(int i)
b.diag(int i, int k1, int k2)
\end{tmvcode}
\index{BandMatrix!methods!nrows}
\index{BandMatrix!methods!ncols}
\index{BandMatrix!methods!rowsize}
\index{BandMatrix!methods!colsize}
\index{BandMatrix!methods!operator()}
\index{BandMatrix!methods!row}
\index{BandMatrix!methods!col}
\index{BandMatrix!methods!diag}
The versions of \tt{row} and \tt{col} with only one argument are
missing, since the full row or column isn't accessible as a \tt{VectorView}.
You must specify a valid range within the row or column that you want, 
given the banded storage of \tt{b}.

\begin{tmvcode}
b.SubVector(int i, int j, int istep, int jstep, int size)
b.SubMatrix(int i1, int i2, int j1, int j2)
b.SubMatrix(int i1, int i2, int j1, int j2, int istep, int jstep)
\end{tmvcode}
\index{BandMatrix!methods!SubVector}
\index{BandMatrix!methods!SubMatrix}
These work the same as for a \tt{Matrix}
(See \ref{matrixaccess}),
except that the entire
subvector or submatrix must be completely within the band.
\label{bandmatrixaccess}

\begin{tmvcode}
b.SubBandMatrix(int i1, int i2, int j1, int j2, int newnlo, int newnhi)
b.SubBandMatrix(int i1, int i2, int j1, int j2, int newnlo, int newnhi, 
      int istep, int jstep)
\end{tmvcode}
\index{BandMatrix!methods!SubBandMatrix}
This returns a \tt{BandMatrixView} of a subset of a \tt{BandMatrix}.
The \tt{newnlo} and \tt{newnhi} parameters do not need to be different
from the existing \tt{nlo} and \tt{nhi} if \tt{i1} = \tt{j1} 
(i.e. the new main diagonal
is part of the old main diagonal).  However, if you are moving the upper left
corner off of the diagonal, you need to adjust \tt{nlo} and \tt{nhi} appropriately.
For example, if \tt{b} is a $6 \times 6$ \tt{BandMatrix} with 2 sub-diagonals and 
3 super-diagonals
(like our example above), the 3 super-diagonals may be viewed as a \tt{BandMatrixView}
with \tt{b.SubBandMatrix(0,5,1,6,0,2)}.

\begin{tmvcode}
b.Rows(int i1,int i2)
b.Cols(int j1,int j2)
b.Diags(int k1, int k2)
\end{tmvcode}
\index{BandMatrix!methods!Rows}
\index{BandMatrix!methods!Cols}
\index{BandMatrix!methods!Diags}
These return a \tt{BandMatrixView} of the parts of these rows,
columns or diagonals that
appear within the original banded structure.  For our example of viewing just
the super-diagonals of a $6 \times 6$ \tt{BandMatrix} with 2 sub- and 
3 super-diagonals, we
could instead use \tt{m.Diags(1,4)}.  The last 3 rows would be \tt{m.Rows(3,6)}.
Note that this wold be a $3 \times 5$ matrix with 0 sub-diagonals and 
4 super-diagonals.
These routines calculate the appropriate changes in the size and shape to include
all of the relevant parts of the rows or columns.

\begin{tmvcode}
b.UpperBand()
b.LowerBand()
\end{tmvcode}
\index{BandMatrix!methods!UpperBand}
\index{BandMatrix!methods!LowerBand}
These return a \tt{BandMatrixView} including the main diagonal and either the
super- or sub-diagonals.  The size
is automatically set appropriately to include the entire band.  (This is only
non-trivial for non-square band matrices.)  They are equivalent to
\tt{b.Diags(0,b.nhi()+1)} and \tt{b.Diags(-b.nlo(),1)} respectively.

\begin{tmvcode}
b.UpperBandOff()
b.LowerBandOff()
\end{tmvcode}
\index{BandMatrix!methods!UpperBandOff}
\index{BandMatrix!methods!LowerBandOff}
These return a \tt{BandMatrixView} of only the off-diagonals of either the
upper or lower half of the matrix.  They are equivalent to
\tt{b.Diags(1,b.nhi()+1)} and \tt{b.Diags(-b.nlo(),0)} respectively.  
They are inspired by analogy with the combination \tt{m.UpperTri().OffDiag()}.
Since \tt{BandMatrix} does not have the method \tt{OffDiag}, these provide
the same functionality.

\begin{tmvcode}
b.Transpose()
b.Conjugate()
b.Adjoint()
b.View()
b.Real()
b.Imag()
\end{tmvcode}
\index{BandMatrix!methods!Transpose}
\index{BandMatrix!methods!Conjugate}
\index{BandMatrix!methods!Adjoint}
\index{BandMatrix!methods!View}
\index{BandMatrix!methods!Real}
\index{BandMatrix!methods!Imag}
\vspace{12pt}

\subsubsection{Functions}
\index{BandMatrix!functions of}

\begin{tmvcode}
RT b.Norm1() = Norm1(b)
RT b.Norm2() = Norm2(b) = b.DoNorm2()
RT b.NormInf() = NormInf(b)
RT b.NormF() = NormF(b) = b.Norm() = Norm(b)
RT b.NormSq() = NormSq(b)
RT b.NormSq(RT scale)
RT b.MaxAbsElement() = MaxAbsElement(b)
T b.Trace() = Trace(b)
T b.Det() = Det(b)
RT b.LogDet(T* sign) = LogDet(b)
bool b.Singular()
RT b.Condition()
RT b.DoCondition()
minv = b.Inverse() = Inverse(b)
b.Inverse(Matrix<T>& minv)
b.InverseATA(Matrix<T>& cov)
\end{tmvcode}
\index{BandMatrix!functions of!Norm1}
\index{BandMatrix!functions of!Norm2}
\index{BandMatrix!functions of!NormInf}
\index{BandMatrix!functions of!MaxAbsElement}
\index{BandMatrix!methods!Norm1}
\index{BandMatrix!methods!Norm2}
\index{BandMatrix!methods!DoNorm2}
\index{BandMatrix!methods!NormInf}
\index{BandMatrix!methods!MaxAbsElement}
\index{BandMatrix!functions of!Norm}
\index{BandMatrix!functions of!NormF}
\index{BandMatrix!functions of!NormSq}
\index{BandMatrix!functions of!Trace}
\index{BandMatrix!functions of!Det}
\index{BandMatrix!functions of!LogDet}
\index{BandMatrix!functions of!Inverse}
\index{BandMatrix!methods!Norm}
\index{BandMatrix!methods!NormF}
\index{BandMatrix!methods!NormSq}
\index{BandMatrix!methods!Trace}
\index{BandMatrix!methods!Det}
\index{BandMatrix!methods!LogDet}
\index{BandMatrix!methods!Singular}
\index{BandMatrix!methods!Condition}
\index{BandMatrix!methods!DoCondition}
\index{BandMatrix!methods!Inverse}
\index{BandMatrix!methods!InverseATA}
The inverse of a \tt{BandMatrix} is not (in general) banded.  So \tt{minv} here
must be a regular \tt{Matrix}.

\begin{tmvcode}
b.Zero()
b.SetAllTo(T x)
b.Clip(RT thresh)
b.SetToIdentity(T x = 1)
b.ConjugateSelf()
b.TransposeSelf()
Swap(b1,b2)
\end{tmvcode}
\index{BandMatrix!methods!Zero}
\index{BandMatrix!methods!SetAllTo}
\index{BandMatrix!methods!Clip}
\index{BandMatrix!methods!SetToIdentity}
\index{BandMatrix!methods!ConjugateSelf}
\index{BandMatrix!methods!TransposeSelf}
\index{BandMatrix!functions of!Swap}
\vspace{12pt}

\subsubsection{Arithmetic}
\index{BandMatrix!arithmetic}

In addition to \tt{x}, \tt{v}, and \tt{m} from before, we now add \tt{b} 
for a \tt{BandMatrix}.

\begin{tmvcode}
b2 = -b1
b2 = x * b1
b2 = b1 [*/] x
b3 = b1 [+-] b2
m2 = m1 [+-] b
m2 = b [+-] m1
b [*/]= x
b2 [+-]= b1
m [+-]= b
v2 = b * v1
v2 = v1 * b
v *= b
b3 = b1 * b2
m2 = b * m1
m2 = m1 * b
m *= b
b2 = b1 [+-] x
b2 = x [+-] b1
b [+-]= x
b = x
\end{tmvcode}

\subsubsection{Division}
\index{BandMatrix!arithmetic!division}

The division operations are:
\begin{tmvcode}
v2 = v1 [/%] b
m2 = m1 [/%] b
m2 = b [/%] m1
m = b1 [/%] b2
m = x [/%] b
v [/%]= b
m [/%]= b
\end{tmvcode}
\tt{BandMatrix} has three possible choices for the division decomposition:
\begin{enumerate}
\item
\tt{b.DivideUsing(tmv::LU)} does a normal LU decomposition,
taking into account the band structure of the matrix, which greatly speeds up 
the calculation into the lower and upper (banded) triangles.
This is the default decomposition to use for a square \tt{BandMatrix} 
if you don't specify anything.

This decomposition can only really
be done in place if either \tt{nlo} or \tt{nhi} is 0,
in which case it is automatically done in place,
since the \tt{BandMatrix} is already lower or upper triangle.
Thus, there is usually no reason to use the \tt{DivideInPlace()} method.

If this is not the case, and you really want to do the decomposition in place, you can
declare a matrix with a wider band and view the portion that represents the matrix
you actually want.  This view then can be divided in place.  More specifically,
you need to declare the wider \tt{BandMatrix} with \tt{ColMajor} storage,
with the smaller of
\{\tt{nlo}, \tt{nhi}\} as the number of sub-diagonals, and with 
(\tt{nlo} + \tt{nhi}) as the number of super-diagonals.  
Then you can use \tt{BandMatrixViewOf} to view the portion you want, transposing it if
necessary.  On the other hand, you are probably not going to get much of a speed gain 
from all of this finagling, so unless you are really memory starved, it's probably not worth it.

To access this decomposition, use:
\begin{tmvcode}
bool b.LUD().IsTrans()
tmv::LowerTriMatrix<T,UnitDiag> b.LUD().GetL()
tmv::ConstBandMatrixView<T> b.LUD().GetU()
int* b.LUD().GetP()
\end{tmvcode}
\index{BandMatrix!methods!LUD}
\index{BandMatrix!LU decomposition}
\index{LU Decomposition!BandMatrix}
The following should result in a matrix numerically very close to \tt{b}.
\begin{tmvcode}
tmv::Matrix<T> m2 = b.LUD().GetL() * b.LUD().GetU();
m2.ReversePermuteRows(b.LUD().GetP());
if (b.LUD().IsTrans()) m2.TransposeSelf();
\end{tmvcode}

\item
\tt{b.DivideUsing(tmv::QR)} will perform a QR decomposition.  
This is the default method for a non-square \tt{BandMatrix}.

The same kind of convolutions need to be done to perform this in place as 
for the LU decomposition.

To access this decomposition, use:\footnote{
I have not yet made a version of the \tt{PackedQ} class for \tt{BandMatrix}.
So unfortunately, here \tt{GetQ()} creates the matrix directly and is thus
rather inefficient.}
\begin{tmvcode}
bool b.QRD().IsTrans()
tmv::Matrix<T> b.QRD().GetQ()
tmv::ConstBandMatrixView<T> b.QRD().GetR()
\end{tmvcode}
\index{BandMatrix!methods!QRD}
\index{BandMatrix!QR decomposition}
\index{QR Decomposition!BandMatrix}
The following should result in a matrix numerically very close to \tt{b}.
\begin{tmvcode}
tmv::Matrix<T> m2(b.nrows,b.ncols);
tmv::MatrixView<T> m2v = 
      b.QRD().IsTrans() ? b2.Transpose() : b2.View();
m2v = b.QRD().GetQ() * b.QRD().GetR();
\end{tmvcode}

\item
\tt{b.DivideUsing(tmv::SV)} will perform a singular value decomposition.

This cannot be done in place.

To access this decomposition, use:
\begin{tmvcode}
tmv::ConstMatrixView<T> b.SVD().GetU()
tmv::ConstDiagMatrixView<RT> b.SVD().GetS()
tmv::ConstMatrixView<T> b.SVD().GetV()
\end{tmvcode}
\index{BandMatrix!methods!SVD}
\index{BandMatrix!singular value decomposition}
\index{Singular Value Decomposition!BandMatrix}
The product of these three
should result in a matrix numerically very close to \tt{b}.

There are the same control and access routines as for a regular SVD
(See \ref{matrixsvd}),
\begin{tmvcode}
b.SVD().Thresh(RT thresh)
b.SVD().Top(int nsing)
RT b.SVD().Condition()
int b.SVD().GetKMax()
\end{tmvcode}

\end{enumerate}
The routines 
\begin{tmvcode}
b.SaveDiv()
b.SetDiv()
b.ReSetDiv()
b.UnSetDiv()
bool b.DivIsSet()
b.DivideInPlace()
\end{tmvcode}
\index{BandMatrix!methods!SaveDiv}
\index{BandMatrix!methods!SetDiv}
\index{BandMatrix!methods!ReSetDiv}
\index{BandMatrix!methods!UnSetDiv}
\index{BandMatrix!methods!DivIsSet}
\index{BandMatrix!methods!DivideInPlace}
work the same as for regular \tt{Matrix}es.
(See \ref{matrixsetdiv}.)

\subsubsection{Input/Output}
\index{BandMatrix!input/output}

The simplest output is the usual:
\begin{tmvcode}
os << b
\end{tmvcode}
where \tt{os} is any \tt{std::ostream}.
The output format is the same as for a \tt{Matrix}.
(See \ref{matrixio}.)

There is also a compact format for a \tt{BandMatrix}:
\begin{tmvcode}
b.WriteCompact(os)
\end{tmvcode}
\index{BandMatrix!methods!WriteCompact}
outputs in the format:
\begin{tmvcode}
B nrows ncols nlo nhi
( b(0,0)  b(0,1)  b(0,2) ... b(0,nhi) )
( b(1,0)  b(1,1)  b(1,2) ... b(1,nhi+1) )
...
( b(nlo,0)  b(nlo,1) ...  b(nlo,nlo+nhi) )
...
( b(nrows-nhi-1,nrows-nlo-nhi-1) ... b(nrows-nhi-1,ncols-1) )
...
( b(nrows-1,nrows-nlo-1)  ... b(nrows-1,ncols-1) )
\end{tmvcode}
If \tt{nrows} is not equal to \tt{ncols}, then the above isn't exactly accurate.  But the
essence is the same: all the values in the band from each row are output one row at a time.

The same compact format can be read back in the usual two ways:
\begin{tmvcode}
tmv::BandMatrix<T> b(nrows,ncols,nlo,nhi);
is >> b;
std::auto_ptr<tmv::BandMatrix<T> > pb;
is >> pb;
\end{tmvcode}

One can write small values as 0 with
\begin{tmvcode}
b.Write(std::ostream& os, RT thresh)
b.WriteCompact(std::ostream& os, RT thresh)
\end{tmvcode}
\index{BandMatrix!methods!WriteCompact}
\index{BandMatrix!methods!Write}

\subsection{Symmetric and hermitian matrices}
\index{SymMatrix}
\index{HermMatrix|see{SymMatrix}}

The \tt{SymMatrix} class is our symmetric matrix class.  A symmetric matrix is
one for which $m = m^T$.  We also have a class called
\tt{HermMatrix}, which is our hermitian matrix class.  A hermitian matrix
is one for which $m = m^\dagger$.  The two are exactly the same 
if \tt{T} is real, but for complex \tt{T}, they are different.

Both classes inherit from \tt{GenSymMatrix},
which has an internal parameter to keep track
of whether it is symmetric or hermitian.
\tt{GenSymMatrix} in turn inherits from \tt{BaseMatrix}.  
The various views and composite classes described below 
also inherit from \tt{GenSymMatrix}.
\index{SymMatrix!SymMatrixView}
\index{SymMatrix!ConstSymMatrixView}
\index{SymMatrix!GenSymMatrix}
\index{SymMatrix!SymMatrixComposite}
\index{BaseMatrix}

One general caveat about complex \tt{HermMatrix} calculations is that the diagonal
elements should all be real.  Some calculations assume this, and only 
use the real part of the diagonal elements.  Other calculations use the 
full complex value as it is in memory.  Therefore, if you set a diagonal element 
to a non-real value at some point, the results will likely be wrong in
unpredictable ways.  Plus of course, your matrix won't actually be hermitian any more,
so the right answer is undefined in any case.

All the \tt{SymMatrix} and \tt{HermMatrix} routines are included by:
\begin{tmvcode}
#include "TMV_Sym.h"
\end{tmvcode}
\index{TMV\_Sym.h}

In addition to the \tt{T} template parameter, there are three other template 
parameters: \tt{uplo}, which can be either \tt{tmv::Upper} or \tt{tmv::Lower};
\tt{stor}, which can be either \tt{tmv::RowMajor} or \tt{tmv::ColMajor}; and
\tt{index}, which can be either \tt{tmv::CStyle} or \tt{tmv::FortranStyle}.
The parameter \tt{uplo} refers to which triangle the data are actually stored in, 
since the
other half of the values are identical, so we do not need to reference them.
The default values for these are \tt{Lower}, \tt{ColMajor}, and \tt{CStyle}
respectively.

The storage of a \tt{SymMatrix} takes
$N \times N$ elements of memory, even though approximately half of them 
are never used.  Someday, I'll write the packed storage versions, which allow for
efficient storage of the matrices.

Usually, the symmetric/hermitian distinction does not affect the use of the classes.
(It does affect the actual calculations performed of course.)  So we will use 
\tt{s} for both, and just point out whenever a \tt{HermMatrix} acts differently
from a \tt{SymMatrix}.

\subsubsection{Constructors}
\index{SymMatrix!constructors}

\begin{itemize}
\item 
\begin{tmvcode}
tmv::SymMatrix<T,uplo,stor,index> s(size_t n)
tmv::HermMatrix<T,uplo,stor,index> s(size_t n)
\end{tmvcode}
Makes an \tt{n} $\times$ \tt{n} \tt{SymMatrix} or \tt{HermMatrix}
with \underline{uninitialized} values.
If debugging is turned on (i.e. not turned off
with -DNDEBUG), then the values are in fact initialized to 888.  

\item
\begin{tmvcode}
tmv::SymMatrix<T,uplo,stor,index> s(size_t n, T x)
tmv::HermMatrix<T,uplo,stor,index> s(size_t n, RT x)
\end{tmvcode}
Makes an \tt{n} $\times$ \tt{n} \tt{SymMatrix} or \tt{HermMatrix} 
with all values equal to \tt{x}.
For the \tt{HermMatrix} version of this, \tt{x} must be real.

\item
\begin{tmvcode}
tmv::SymMatrix<T,uplo,stor,index> s(size_t n, const T* vv)
tmv::SymMatrix<T,uplo,stor,index> s(size_t n, 
      const std::vector<T>& vv)
tmv::HermMatrix<T,uplo,stor,index> s(size_t n, const T* vv)
tmv::HermMatrix<T,uplo,stor,index> s(size_t n, 
      const std::vector<T>& vv)
\end{tmvcode}
Makes a \tt{SymMatrix} or \tt{HermMatrix} which copies the elements from \tt{vv}.

\item 
\begin{tmvcode}
tmv::SymMatrix<T,uplo,stor,index> s(const GenMatrix<T>& m)
tmv::HermMatrix<T,uplo,stor,index> s(const GenMatrix<T>& m)
tmv::SymMatrix<T,uplo,stor,index> s(const GenDiagMatrix<T>& d)
tmv::HermMatrix<T,uplo,stor,index> s(const GenDiagMatrix<T>& d)
\end{tmvcode}
Makes a \tt{SymMatrix} or \tt{HermMatrix} which copies the corresponding values of \tt{m}.

\item
\begin{tmvcode}
tmv::SymMatrix<T,uplo,stor,index> s1(const GenSymMatrix<T2>& s2)
tmv::HermMatrix<T,uplo,stor,index> s1(const GenSymMatrix<T2>& s2)
s1 = s2
\end{tmvcode}
Copy the \tt{GenSymMatrix s2}, which may be of any type \tt{T2} so long
as values of type \tt{T2} are convertible into type \tt{T}.
The assignment operator has the same flexibility.
If \tt{T} and \tt{T2} are complex, then \tt{m1} and \tt{m2} need to be
either both symmetric or both hermitian.

\item
\begin{tmvcode}
tmv::SymMatrixView<T,index> s = 
      tmv::SymMatrixViewOf(MatrixView<T,index> m, UpLoType uplo)
tmv::ConstSymMatrixView<T,index> s = 
      tmv::SymMatrixViewOf(ConstMatrixView<T,index> m, UpLoType uplo)
tmv::SymMatrixView<T,index> s = 
      tmv::HermMatrixViewOf(MatrixView<T,index>& m, UpLoType uplo)
tmv::ConstSymMatrixView<T,index> s = 
      tmv::HermMatrixViewOf(ConstMatrixView<T,index>& m, UpLoType uplo)
\end{tmvcode}
\index{Matrix!view portion as SymMatrix or HermMatrix}
Make a \tt{SymMatrixView} of the corresponding portion of \tt{m}.

\item
\begin{tmvcode}
tmv::SymMatrixView<T,index> s = 
      tmv::SymMatrixViewOf(T* vv, size_t n, UpLoType uplo, 
      StorageType stor)
tmv::ConstSymMatrixView<T,index> s =
      tmv::SymMatrixViewOf(const T* vv, size_t n, UpLoType uplo, 
      StorageType stor)
tmv::SymMatrixView<T,index> s =
      tmv::HermMatrixViewOf(T* vv, size_t n, UpLoType uplo, 
      StorageType stor)
tmv::ConstSymMatrixView<T,index> s =
      tmv::HermMatrixViewOf(const T* vv, size_t n, UpLoType uplo, 
      StorageType stor)
\end{tmvcode}
\index{SymMatrix!view of raw memory}
Make a \tt{SymMatrixView} of the actual memory elements, \tt{vv}, in either the 
upper or lower triangle.
\tt{vv} must be of length \tt{n} $\times$ \tt{n}, even though only about half 
of the values are actually used,

\end{itemize}

\subsubsection{Access}
\index{SymMatrix!access methods}

\begin{tmvcode}
s.nrows() = s.ncols() = s.colsize() = s.rowsize() = s.size()
s(i,j)
s.row(int i, int j1, int j2)
s.col(int i, int j1, int j2)
s.diag()
s.diag(int i)
s.diag(int i, int k1, int k2)
\end{tmvcode}
\index{SymMatrix!methods!nrows}
\index{SymMatrix!methods!ncols}
\index{SymMatrix!methods!rowsize}
\index{SymMatrix!methods!colsize}
\index{SymMatrix!methods!size}
\index{SymMatrix!methods!operator()}
\index{SymMatrix!methods!row}
\index{SymMatrix!methods!col}
\index{SymMatrix!methods!diag}
For the mutable \tt{d(i,j)} version, 
As for triangle matrices, the versions of \tt{row} and \tt{col} with only one argument are
missing, since the full row or column isn't accessible as a \tt{VectorView}.
You must specify a valid range within the row or column that you want, 
given the upper or lower storage of \tt{s}.
The diagonal element may be in a \tt{VectorView} with either elements in the 
lower triangle or the upper triangle, but not both.  To access a full row, you would 
therefore need to use two steps:
\begin{tmvcode}
s.row(i,0,i) = ...
s.row(i,i,ncols) = ...
\end{tmvcode}

\begin{tmvcode}
s.SubVector(int i, int j, int istep, int jstep, int size)
s.SubMatrix(int i1, int i2, int j1, int j2)
s.SubMatrix(int i1, int i2, int j1, int j2, int istep, int jstep)
\end{tmvcode}
\index{SymMatrix!methods!SubVector}
\index{SymMatrix!methods!SubMatrix}
These work the same as for a \tt{Matrix}
(See \ref{matrixaccess}.)
except that the entire
subvector or submatrix must be completely within the upper or lower triangle.

\begin{tmvcode}
s.SubSymMatrix(int i1, int i2, int istep = 1)
\end{tmvcode}
\index{SymMatrix!methods!SubSymMatrix}
This returns a \tt{SymMatrixView} of \tt{s} whose upper-left
corner is \tt{s(i1,i1)}, and whose lower-right corner is 
\tt{s(i2-istep,i2-istep)}.  If \tt{istep} $\neq 1$, then it is the 
step in both the \tt{i} and \tt{j} directions.

\begin{tmvcode}
s.UpperTri(DiagType dt=NonUnitDiag)
s.LowerTri(DiagType dt=NonUnitDiag)
\end{tmvcode}
\index{SymMatrix!methods!UpperTri}
\index{SymMatrix!methods!LowerTri}
Both of these are valid, regardless
of which triangle stores the actual data for \tt{s}.
\begin{tmvcode}
s.Transpose()
s.Conjugate()
s.Adjoint()
s.View()
s.Real()
s.Imag()
\end{tmvcode}
\index{SymMatrix!methods!Transpose}
\index{SymMatrix!methods!Conjugate}
\index{SymMatrix!methods!Adjoint}
\index{SymMatrix!methods!View}
\index{SymMatrix!methods!Real}
\index{SymMatrix!methods!Imag}
Note that the imaginary part of a complex hermitian matrix is skew-symmetric,
so \tt{s.Imag()} is illegal for a \tt{HermMatrix}.  If you need to
deal with the imaginary part of a \tt{HermMatrix},
you would have to view them as a triangle matrix with \tt{s.UpperTri().Imag()}.  Or, 
since the diagonal elements are all real.
you could also use \tt{s.UpperTri().OffDiag().Imag()}. 
\vspace{12pt}

\subsubsection{Functions}
\index{SymMatrix!functions of}

\begin{tmvcode}
RT s.Norm1() = Norm1(s)
RT s.Norm2() = Norm2(s) = s.DoNorm2()
RT s.NormInf() = NormInf(s)
RT s.NormF() = NormF(s) = s.Norm() = Norm(s)
RT s.NormSq() = NormSq(s)
RT s.NormSq(RT scale)
RT s.MaxAbsElement() = MaxAbsElement(s)
T s.Trace() = Trace(s)
T s.Det() = Det(s)
RT s.LogDet(T* sign=0) = LogDet(s)
bool s.Singular()
RT s.Condition()
RT s.DoCondition()
sinv = s.Inverse() = Inverse(s)
s.Inverse(Matrix<T>& sinv)
s.Inverse(SymMatrix<T>& sinv)
s.InverseATA(Matrix<T>& cov)
\end{tmvcode}
\index{SymMatrix!functions of!Norm1}
\index{SymMatrix!functions of!Norm2}
\index{SymMatrix!functions of!NormInf}
\index{SymMatrix!functions of!MaxAbsElement}
\index{SymMatrix!methods!Norm1}
\index{SymMatrix!methods!Norm2}
\index{SymMatrix!methods!DoNorm2}
\index{SymMatrix!methods!NormInf}
\index{SymMatrix!methods!MaxAbsElement}
\index{SymMatrix!functions of!Norm}
\index{SymMatrix!functions of!NormF}
\index{SymMatrix!functions of!NormSq}
\index{SymMatrix!functions of!Trace}
\index{SymMatrix!functions of!Det}
\index{SymMatrix!functions of!LogDet}
\index{SymMatrix!functions of!Inverse}
\index{SymMatrix!methods!Norm}
\index{SymMatrix!methods!NormF}
\index{SymMatrix!methods!NormSq}
\index{SymMatrix!methods!Trace}
\index{SymMatrix!methods!Det}
\index{SymMatrix!methods!LogDet}
\index{SymMatrix!methods!Singular}
\index{SymMatrix!methods!Condition}
\index{SymMatrix!methods!DoCondition}
\index{SymMatrix!methods!Inverse}
\index{SymMatrix!methods!InverseATA}
Since the inverse of an \tt{SymMatrix} is also symmetric,
the object returned by \tt{s.Inverse()} is 
assignable to a \tt{SymMatrix}.  Of course you can also assign it
to a regular \tt{Matrix} if you prefer.  Similarly, there are versions
of \tt{s.Inverse(minv)} for both argument types.  

\begin{tmvcode}
s.Zero()
s.SetAllTo(T x)
s.Clip(RT thresh)
s.SetToIdentity(T x = 1)
s.ConjugateSelf()
s.TransposeSelf()
Swap(s1,s2)
\end{tmvcode}
\index{SymMatrix!methods!Zero}
\index{SymMatrix!methods!SetAllTo}
\index{SymMatrix!methods!Clip}
\index{SymMatrix!methods!SetToIdentity}
\index{SymMatrix!methods!ConjugateSelf}
\index{SymMatrix!methods!TransposeSelf}
\index{SymMatrix!functions of!Swap}
\tt{TransposeSelf} does nothing to a \tt{SymMatrix} and is equivalent to
\tt{ConjugateSelf} for a \tt{HermMatrix}.
\begin{tmvcode}
s.SwapRowsCols(int i1, int i2)
s.PermuteRowsCols(const int* p)
s.ReversePermuteRowsCols(const int* p)
s.PermuteRowsCols(const int* p, int i1, int i2)
s.ReversePermuteRowsCols(const int* p, int i1, int i2)
\end{tmvcode}
\index{SymMatrix!methods!SwapRowsCols}
\index{SymMatrix!methods!PermuteRowsCols}
\index{SymMatrix!methods!ReversePermuteRowsCols}
The new method, \tt{SwapRowsCols}, would be equivalent to 
\begin{tmvcode}
s.SwapRows(i1,i2).SwapCols(i1,i2);
\end{tmvcode}
except that neither of these functions are allowed for a \tt{SymMatrix}, since 
they result in non-symmetric matrices.  Only the combination of both
maintains the symmetry of the matrix.  So this combination is included as
a method.  The permute methods performs a series of these
combined swaps.
\vspace{12pt}

\subsubsection{Arithmetic}
\index{SymMatrix!arithmetic}
\label{symarith}

In addition to \tt{x}, \tt{v}, and \tt{m} from before,
we now add \tt{s} for a \tt{SymMatrix}.

\begin{tmvcode}
s2 = -s1
s2 = x * s1
s2 = s1 [*/] x
s3 = s1 [+-] s2
m2 = m1 [+-] s
m2 = s [+-] m1
s [*/]= x
s2 [+-]= s1
m [+-]= s
v2 = s * v1
v2 = v1 * s
v *= s
m = s1 * s2
m2 = s * m1
m2 = m1 * s
m *= s
s2 = s1 [+-] x
s2 = x [+-] s1
s [+-]= x
s = x
s = v ^ v
s [+-]= v ^ v
s = m * m.Transpose()
s [+-]= m * m.Transpose()
s = U * U.Transpose()
s [+-]= U * U.Transpose()
s = L * L.Transpose()
s [+-]= L * L.Transpose()
\end{tmvcode}
\index{Vector!arithmetic!outer product}
\index{SymMatrix!arithmetic!rank-1 update}
\index{SymMatrix!arithmetic!rank-k update}
For outer products, both \tt{v}'s need to be the same actual data.  If \tt{s}
is complex hermitian, then it should actually be 
%\tt{s = v \caret v.Conjugate()}.
\tt{s = v ^ v.Conjugate()}.
Likewise for the next three (called ``rank-k updates''), the \tt{m}'s, \tt{L}'s and
\tt{U}'s need to be the
same data, and for a complex hermitian matrix, \tt{Transpose()}
should be replaced with \tt{Adjoint()}.

There is a minor issue with \tt{SymMatrix} arithmetic that is
worth knowing about.
Because TMV uses the same base class for both hermitian and symmetric
matrices, the compiler cannot tell the difference between them in arithmetic
operations.  
This can become an issue when doing complicated arithmetic operations
with complex hermitian or symmetric matrices.

Some operations with a \tt{GenSymMatrix}, as far as the compiler can tell, may or may
not result in another \tt{SymMatrix}.  For example, a complex scalar times a complex 
\tt{HermMatrix} is not hermitian (or symmetric), but a complex scalar times a \tt{SymMatrix}
is still symmetric.
Likewise the sum of two \tt{GenSymMatrix}es may or may not be hermitian or symmetric.

So the complex \tt{SymMatrixComposite} class\footnote{
Note: all of these comments only apply to \underline{complex} \tt{SymMatrix} and \tt{HermMatrix}
arithmetic.  The real version of the \tt{SymMatrixComposite} does derive from
\tt{GenSymMatrix}, since there is no ambiguity in that case.},
which is the return type of these and similar operations, 
is derived from \tt{GenMatrix}
rather than from \tt{GenSymMatrix}.  This means that if you let it
self-instantiate, rather than assign it to a \tt{SymMatrix}, it will 
instantiate as a regular \tt{Matrix}.  However, it is assignable to a \tt{SymMatrix},
despite deriving from \tt{GenMatrix},
so if your operation should be allowed, then the assignment will work fine.

Most of the time, this won't matter, since
you will generally assign the result to either a \tt{SymMatrix} or a \tt{Matrix}
as appropriate.  However, if you let your expression get more complicated than a single
matrix addition, multiplication, etc., then some things that should be allowed give compiler errors.
For example:
\begin{tmvcode}
s3 += x*s1+s2;
\end{tmvcode}
is not legal for complex symmetric \tt{s1, s2, s3}, even though this is valid mathematically.
This is because there is no 3-matrix 
\tt{Add} function.  (The \tt{AddMM} function just adds a multiple of one 
matrix to another.)  So the right hand side needs to be instantiated before
being added to the left side, and it will instantiate as a regular \tt{Matrix},
which cannot be added to a \tt{SymMatrix}.  
If the ``\tt{+=}'' had been just ``\tt{=}'', then we wouldn't have any problem,
since the composite object that stores \tt{x*s1+s2} is assignable to a \tt{SymMatrix}.

One work-around is to explicitly tell the compiler to instantiate the right hand
side as a \tt{SymMatrix}:
\begin{tmvcode}
s3 += SymMatrix<T>(x*s1+s2);
\end{tmvcode}

Another work-around, which I suspect will usually be preferred, is to break the 
equation into multiple statements, each of which are simple enough to not require
any instantiation:
\begin{tmvcode}
s3 += x*s1;
s3 += s2;
\end{tmvcode}

\subsubsection{Division}
\index{SymMatrix!arithmetic!division}
The division operations are:
\begin{tmvcode}
v2 = v1 [/%] s
m2 = m1 [/%] s
m2 = s [/%] m1
m = s1 [/%] s2
s1 = x [/%] s2
v [/%]= s
m [/%]= s
\end{tmvcode}

\tt{SymMatrix} has three possible choices for the decomposition to use for division:
\begin{enumerate}
\item
\tt{m.DivideUsing(tmv::LU)} will perform something similar to the LU decomposition
for regular matrices.  But in fact, it does what is called an LDL or Bunch-Kaufman
decomposition.  

A permutation of \tt{m} is decomposed into a lower triangle matrix ($L$)
times a symmetric block diagonal matrix ($D$) times the transpose of $L$.
$D$ has either 1x1 and 2x2 blocks down the diagonal.  For hermitian matrices,
the third term is the adjoint of $L$ rather than the transpose.

This is the default decomposition to use if you don't specify anything.

To access this decomposition, use:
\begin{tmvcode}
ConstLowerTriMatrixView<T> s.LUD().GetL()
Matrix<T> s.LUD().GetD()
int* s.LUD().GetP()
\end{tmvcode}
\index{SymMatrix!methods!LUD}
\index{SymMatrix!LU Decomposition|see{SymMatrix!Bunch-Kaufman decomposition}}
\index{SymMatrix!Bunch-Kaufman decomposition}
\index{Bunch-Kaufman Decomposition!SymMatrix}
The following should result in a matrix numerically very close to \tt{s}.
\begin{tmvcode}
Matrix<T> m2 = s.LUD().GetL() * s.LUD().GetD() * 
      s.LUD().GetL().Transpose();
m2.ReversePermuteRows(s.LUD().GetP());
m2.ReversePermuteCols(s.LUD().GetP());
\end{tmvcode}
For a complex hermitian \tt{s}, you would need to replace
\tt{Transpose} with \tt{Adjoint}.

\item
\tt{s.DivideUsing(tmv::CH)} will perform a Cholesky decomposition.  
The matrix \tt{s} must be hermitian (or real symmetric) to use \tt{CH}, since that is the
only kind of matrix that has a Cholesky decomposition.  

It is also similar to an 
LU decomposition, where $U$ is the adjoint of $L$, and there is no permutation.
It can be a bit dangerous, since not all hermitian matrices have such a decomposition,
so the decomposition could fail.  Only so-called ``positive-definite'' hermitian 
matrices have a Cholesky decomposition.  A positive-definite matrix has
all positive real eigenvalues.  In general, hermitian matrices have real, but
not necessarily positive eigenvalues.  

One example of a 
positive-definite matrix is $s = A^\dagger A$ where $A$ is any matrix.   
Then $s$ is guaranteed to be positive-semi-definite
(which means some of the eigenvalues may be 0, but not negative).
In this case, the routine will usually work, but still might fail from 
numerical round-off errors if $s$ is nearly singular.  

When the decomposition fails, it throws an object of type
\tt{NonPosDef}.
\index{Exceptions!NonPosDef}

See \S\ref{nonposdef} for some more discussion about positive-definite
matrices.

The only advantage of Cholesky over Bunch-Kaufman is speed.  (And only about
20 to 30\% at that.)  If you know your 
matrix is positive-definite, the Cholesky decomposition is the fastest way to 
do division.

To access this decomposition, use:
\begin{tmvcode}
ConstLowerTriMatrixView<T> s.CHD().GetL()
\end{tmvcode}
\index{SymMatrix!methods!CHD}
\index{SymMatrix!Cholesky decomposition}
\index{Cholesky Decomposition!SymMatrix}
The following should result in a matrix numerically very close to \tt{s}.
\begin{tmvcode}
Matrix<T> m2 = s.CHD().GetL() * s.CHD().GetL().Adjoint()
\end{tmvcode}

\item
\tt{s.DivideUsing(tmv::SV)} will perform either an eigenvalue decomposition
(for hermitian and real symmetric matrices) or a regular singular value
decomposition (for complex symmetric matrices).

For hermitian matrices (including real symmetric matrices), 
the eigenvalue decomposition is $H = U S U^\dagger$, where $U$ is
unitary and $S$ is diagonal.  So this would be identical to a singular
value decomposition where $V = U^\dagger$, 
except that the elements of $S$, the eigenvalues of $H$, may be negative.

However, this decomposition is just as useful for division, dealing with singular
matrices just as elegantly.  It just means that internally, we allow the values
of S to be negative, taking the absolute value when necessary (e.g. for Norm2).
The below access commands finish the calculation of S,V so that the S(i)
values are positive. 

To access this decomposition, use:
\begin{tmvcode}
ConstMatrixView<T> s.SVD().GetU()
DiagMatrix<RT> s.SVD().GetS()
Matrix<T> s.SVD().GetV()
\end{tmvcode}
\index{SymMatrix!methods!SVD}
\index{SymMatrix!methods!SymSVD}
\index{SymMatrix!singular value decomposition}
\index{Singular Value Decomposition!SymMatrix}
The following should result in a matrix numerically very close to \tt{s}.
\begin{tmvcode}
Matrix<T> m2 = s.SVD().GetU() * s.SVD().GetS() * s.SVD().GetV()
\end{tmvcode}

For a complex symmetric \tt{s}, the situation is not as convenient.
In principle, one could find a decomposition $s = USU^T$ where $S$ is 
a complex diagonal matrix, but such a decomposition is not easy to find. 
So for complex symmetric matrices, we
just do the normal SVD: $s = USV$, although the algorithm
does use the symmetry of the matrix to 
speed up portions of the algorithm relative that that for a general matrix.

The access is also necessarily different, since the object returned by 
\tt{s.SVD()} implicitly assumes that \tt{V = U.Adjoint()} (modulo some sign changes), 
so we need a 
new accessor: \tt{s.SymSVD()}.  Its \tt{GetS} and \tt{GetV} methods return Views
rather than instantiated matrices.

Both versions also have the same control and access routines as a regular SVD
(See \ref{matrixsvd}):
\begin{tmvcode}
s.SVD().Thresh(RT thresh)
s.SVD().Top(int nsing)
RT s.SVD().Condition()
int s.SVD().GetKMax()
\end{tmvcode}
(Likewise for \tt{s.SymSVD()}.)

\end{enumerate}
The routines 
\begin{tmvcode}
s.SaveDiv()
s.SetDiv()
s.ReSetDiv()
s.UnSetDiv()
bool s.DivIsSet()
s.DivideInPlace()
\end{tmvcode}
\index{SymMatrix!methods!SaveDiv}
\index{SymMatrix!methods!SetDiv}
\index{SymMatrix!methods!ReSetDiv}
\index{SymMatrix!methods!UnSetDiv}
\index{SymMatrix!methods!DivIsSet}
\index{SymMatrix!methods!DivideInPlace}
work the same as for regular \tt{Matrix}es.
(See \ref{matrixsetdiv}.)

\subsubsection{Input/Output}
\index{SymMatrix!input/output}

The simplest output is the usual:
\begin{tmvcode}
os << s
\end{tmvcode}
where \tt{os} is any \tt{std::ostream}.
The output format is the same as for a \tt{Matrix}.

There is also a compact format for a \tt{SymMatrix}:
\begin{tmvcode}
s.WriteCompact(os)
\end{tmvcode}
\index{SymMatrix!methods!WriteCompact}
outputs in the format:
\begin{tmvcode}
H/S n 
( s(0,0) )
( s(1,0)  s(1,1) )
...
( s(n-1,0)  s(n-1,1) ... s(n-1,n-1) )
\end{tmvcode}
where \tt{H/S} means \underline{either} the character \tt{H} or \tt{S}, which indicates whether 
the matrix is hermitian or symmetric.
In each case, the same compact format can be read back in the usual two ways:
\begin{tmvcode}
tmv::SymMatrix<T> s(n);
tmv::HermMatrix<T> h(n);
is >> s >> h;
std::auto_ptr<tmv::SymMatrix<T> > ps;
std::auto_ptr<tmv::HermMatrix<T> > ph;
is >> ps >> ph;
\end{tmvcode}
One can write small values as 0 with
\begin{tmvcode}
s.Write(std::ostream& os, RT thresh)
s.WriteCompact(std::ostream& os, RT thresh)
\end{tmvcode}
\index{SymMatrix!methods!WriteCompact}
\index{SymMatrix!methods!Write}

\subsection{Symmetric and hermitian band matrices}
\index{SymBandMatrix}
\index{HermBandMatrix|see{SymBandMatrix}}

The \tt{SymBandMatrix} class is our symmetric band matrix, which combines
the properties of SymMatrix and BandMatrix; it has a banded structure and
$m = m^T$.  Likewise \tt{HermBandMatrix} is our hermitian band matrix for
which $m = m^\dagger$.

Both classes inherit from \tt{GenSymBandMatrix}, which in turn inherits from 
\tt{BaseMatrix}.  The various views and composite classes described below 
also inherit from \tt{GenSymBandMatrix}.
\index{SymBandMatrix!SymBandMatrixView}
\index{SymBandMatrix!ConstSymBandMatrixView}
\index{SymBandMatrix!GenSymBandMatrix}
\index{SymBandMatrix!SymBandMatrixComposite}
\index{BaseMatrix}

As with the documentation for \tt{SymMatrix}/\tt{HermMatrix}, the descriptions
below will only be written for \tt{SymBandMatrix} with the implication
that a \tt{HermBandMatrix} has the same functionality, but with the calculations
appropriate for a hermitian matrix, rather than symmetric.

One general caveat about complex \tt{HermBandMatrix} calculations is that 
the diagonal elements should all be real.  Some calculations assume this, and 
only use the real part of the diagonal elements.  Other calculations use the 
full complex value as it is in memory.  Therefore, if you set a diagonal element
to a non-real value at some point, the results will likely be wrong in 
unpredictable ways.  Plus of course, your matrix will not actually be 
hermitian any more, so the right answer is undefined in any case.

All the \tt{SymBandMatrix} and \tt{HermBandMatrix} routines are included by:
\begin{tmvcode}
#include "TMV_SymBand.h"
\end{tmvcode}
\index{TMV\_SymBand.h}

In addition to the \tt{T} template parameter, there are three other template 
parameters: \tt{uplo}, which can be either \tt{tmv::Upper} or \tt{tmv::Lower};
\tt{stor}, which can be one of \tt{tmv::RowMajor}, \tt{tmv::ColMajor}, or 
\tt{tmv::DiagMajor}; and \tt{index}, which can be either \tt{tmv::CStyle}
or \tt{tmv::FortranStyle}.  The default values for these are 
\tt{Lower}, \tt{ColMajor}, \tt{CStyle}.

The storage size required is the same as for the \tt{BandMatrix} of
the upper or lower band portion.
(See \ref{bandmatrixstoragesize}.)
As with square band matrices, 
all three storage methods always need the same amount of memory, so the 
decision about which method to use should generally be based on performance 
considerations rather than memory usage.
The speed of the various matrix operations are different for the different storage 
types.  If the matrix calculation speed is important, it may be worth trying 
all three to see which is fastest for the operations you are using.

Also, as with \tt{BandMatrix}, the storage for \tt{Lower}, \tt{DiagMajor} 
does not start with the upper left element as usual.
Rather, it starts at the start of the lowest sub-diagonal.  
So for the constructors that 
take the matrix information from an array (\tt{T*} or \tt{vector<T>}), 
the start of the array needs to be at the start of the lowest sub-diagonal.  

\subsubsection{Constructors}
\index{SymBandMatrix!constructors}

\begin{itemize}
\item 
\begin{tmvcode}
tmv::SymBandMatrix<T,uplo,stor,index> sb(size_t n, int nlo)
\end{tmvcode}
Makes an \tt{n} $\times$ \tt{n} \tt{SymBandMatrix} with 
\tt{nlo} off-diagonals
and with \underline{uninitialized} values.
If debugging is turned on (i.e. not turned off 
with -DNDEBUG), then the values are initialized to 888.

\item
\begin{tmvcode}
tmv::SymBandMatrix<T,uplo,stor,index> sb(size_t n, int nlo, T x)
\end{tmvcode}
Makes an \tt{n} $\times$ \tt{n} \tt{SymBandMatrix} with \tt{nlo} off-diagonals
and with all values equal to \tt{x}.

\item
\begin{tmvcode}
tmv::SymBandMatrix<T,uplo,stor,index> sb(size_t n, int nlo, 
      const T* vv)
tmv::SymBandMatrix<T,uplo,stor,index> sb(size_t n, int nlo, 
      const std::vector<T>& vv)
\end{tmvcode}
Makes a \tt{SymBandMatrix} which copies the elements from \tt{vv}.  

\item 
\begin{tmvcode}
tmv::SymBandMatrix<T,uplo,stor,index> sb(const GenMatrix<T>& m, 
      int nlo)
tmv::SymBandMatrix<T,uplo,stor,index> sb(const GenSymMatrix<T>& m, 
      int nlo)
tmv::SymBandMatrix<T,uplo,stor,index> sb(const GenBandMatrix<T>& m, 
      int nlo)
tmv::SymBandMatrix<T,uplo,stor,index> sb(const GenSymBandMatrix<T>& m, 
      int nlo)
\end{tmvcode}
Makes a \tt{SymBandMatrix} which copies the corresponding values of \tt{m}.  
For the last two, \tt{nlo} must not be larger than the number of upper
or lower bands in \tt{m}.

\item
\begin{tmvcode}
tmv::SymBandMatrix<T,uplo,DiagMajor> sb = SymTriDiagMatrix(
      const GenVector<T>& v1, const GenVector<T>& v2)
tmv::SymBandMatrix<T,uplo,DiagMajor> sb = SymTriDiagMatrix(
      const GenVector<T>& v1, const GenVector<T>& v2)
tmv::HermBandMatrix<T,uplo,DiagMajor> sb = HermTriDiagMatrix(
      const GenVector<T>& v1, const GenVector<T>& v2, 
      UpLoType uplo)
tmv::HermBandMatrix<T,uplo,DiagMajor> sb = HermTriDiagMatrix(
      const GenVector<T>& v1, const GenVector<T>& v2, 
      UpLoType uplo)
\end{tmvcode}
\index{SymBandMatrix!constructors!SymTriDiagMatrix}
\index{SymBandMatrix!constructors!HermTriDiagMatrix}
Shorthand to create a symmetric tri-diagonal band matrix
if you already have the \tt{Vector}s.  
The main diagonal is \tt{v1} and the off-diagonal is \tt{v2}.

With a \tt{HermTriDiagMatrix}, \tt{v1} should be real, although
it may be either a real-valued \tt{Vector} or a complex-valued
\tt{Vector} whose imaginary components are all zero.
Also, \tt{HermTriDiagMatrix} takes an extra parameter, \tt{uplo}, indicating
whether \tt{v2} should be used as the upper or lower off-diagonal.

\item
\begin{tmvcode}
tmv::SymBandMatrix<T> sb1(const GenSymBandMatrix<T2>& sb2)
sb1 = sb2
\end{tmvcode}
Copy the \tt{SymBandMatrix m2}, which may be of any type \tt{T2} so long
as values of type \tt{T2} are convertible into type \tt{T}.
The assignment operator has the same flexibility.

\item
\begin{tmvcode}
tmv::SymBandMatrixView<T> sb = 
      SymBandMatrixViewOf(MatrixView<T> m, UpLoType uplo, int nlo)
tmv::SymBandMatrixView<T> sb = 
      SymBandMatrixViewOf(SymMatrixView<T> m, int nlo)
tmv::SymBandMatrixView<T> sb = 
      SymBandMatrixViewOf(BandMatrixView<T> m, UpLoType uplo, int nlo)
tmv::SymBandMatrixView<T> sb = 
      HermBandMatrixViewOf(MatrixView<T> m, UpLoType uplo, int nlo)
tmv::SymBandMatrixView<T> sb = 
      HermBandMatrixViewOf(SymMatrixView<T> m, int nlo)
tmv::SymBandMatrixView<T> sb = 
      HermBandMatrixViewOf(BandMatrixView<T> m, UpLoType uplo, int nlo)
\end{tmvcode}
\index{Matrix!view portion as SymBandMatrix or HermBandMatrix}
\index{SymMatrix!view portion as SymBandMatrix or HermBandMatrix}
\index{BandMatrix!view portion as SymBandMatrix or HermBandMatrix}
Make an \tt{SymBandMatrixView} of the corresponding portion of \tt{m}.  
To view these as a hermitian band matrix, use the command,
\tt{HermBandMatrixViewOf} instead.
For the view of a \tt{BandMatrix}, the parameter \tt{nlo} may be 
omitted, in which case either \tt{m.nhi()} or \tt{m.nlo()} is used 
according to whether \tt{uplo} is \tt{Upper} or \tt{Lower} respectively.
There are also \tt{ConstSymBandMatrixView} versions of these.

\item
\begin{tmvcode}
tmv::SymBandMatrixView<T> sb = 
      tmv::SymBandMatrixViewOf(T* vv, size_t n, int nlo, 
          UpLoType uplo, StorageType stor)
tmv::ConstSymBandMatrixView<T> sb = 
      tmv::SymBandMatrixViewOf(const T* vv, size_t n, int nlo, 
          UpLoType uplo, StorageType stor)
tmv::SymBandMatrixView<T> sb = 
      tmv::HermBandMatrixViewOf(T* vv, size_t n, int nlo, 
          UpLoType uplo, StorageType stor)
tmv::ConstSymBandMatrixView<T> sb = 
      tmv::HermBandMatrixViewOf(const T* vv, size_t n, int nlo, 
          UpLoType uplo, StorageType stor)
\end{tmvcode}
\index{SymBandMatrix!view of raw memory}
Make a \tt{SymBandMatrixView} of the actual memory elements, \tt{vv}.

\end{itemize}

\subsubsection{Access}
\index{SymBandMatrix!access methods}

\begin{tmvcode}
sb.nrows() = sb.ncols() = sb.colsize() = sb.rowsize() = sb.size()
sb.nlo() = sb.nhi()
sb(i,j)
sb.row(int i, int j1, int j2)
sb.col(int i, int j1, int j2)
sb.diag()
sb.diag(int i)
sb.diag(int i, int k1, int k2)
\end{tmvcode}
\index{SymBandMatrix!methods!nrows}
\index{SymBandMatrix!methods!ncols}
\index{SymBandMatrix!methods!rowsize}
\index{SymBandMatrix!methods!colsize}
\index{SymBandMatrix!methods!size}
\index{SymBandMatrix!methods!operator()}
\index{SymBandMatrix!methods!row}
\index{SymBandMatrix!methods!col}
\index{SymBandMatrix!methods!diag}
For the mutable \tt{d(i,j)} version, 
Again, the versions of \tt{row} and \tt{col} with only one argument are
missing, since the full row or column isn't accessible as a \tt{VectorView}.
You must specify a valid range within the row or column that you want, 
given the banded storage of \tt{sb}.  And, like for \tt{SymMatrix}, a full row
must be accessed in its two parts, one on each side of the diagonal.

\begin{tmvcode}
sb.SubVector(int i, int j, int istep, int jstep, int size)
sb.SubMatrix(int i1, int i2, int j1, int j2)
sb.SubMatrix(int i1, int i2, int j1, int j2, int istep, int jstep)
sb.SubBandMatrix(int i1, int i2, int j1, int j2, int newnlo, int newhi)
sb.SubBandMatrix(int i1, int i2, int j1, int j2, int newnlo, int newhi, 
      int istep, int jstep)
sb.Diags(int k1, int k2)
sb.UpperBand()
sb.LowerBand()
sb.UpperBandOff()
sb.LowerBandOff()
\end{tmvcode}
\index{SymBandMatrix!methods!SubVector}
\index{SymBandMatrix!methods!SubMatrix}
\index{SymBandMatrix!methods!SubBandMatrix}
\index{SymBandMatrix!methods!Diags}
\index{SymBandMatrix!methods!UpperBand}
\index{SymBandMatrix!methods!LowerBand}
\index{SymBandMatrix!methods!UpperBandOff}
\index{SymBandMatrix!methods!LowerBandOff}
These work the same as for a \tt{BandMatrix}
(See \ref{bandmatrixaccess}),
except that the entire
subvector or submatrix must be completely within the upper or lower band.


\begin{tmvcode}
sb.SubSymMatrix(int i1, int i2)
sb.SubSymMatrix(int i1, int i2, int istep)
sb.SubSymBandMatrix(int i1, int i2, int newnlo=m.nlo())
sb.SubSymBandMatrix(int i1, int i2, int newnlo, int istep)
\end{tmvcode}
\index{SymBandMatrix!methods!SubSymMatrix}
\index{SymBandMatrix!methods!SubSymBandMatrix}
These return a view of the \tt{SymMatrix} or \tt{SymBandMatrix} which runs
from \tt{i1} to \tt{i2} along the diagonal with an optional step,
and includes the off-diagonals in the same rows/cols.  For the first two,
the \tt{SymMatrix} must be completely with the band.

\begin{tmvcode}
sb.SymDiags(int newnlo)
\end{tmvcode}
\index{SymBandMatrix!methods!SymDiags}
Since \tt{Diags} returns a regular \tt{BandMatrixView}, it must be completely
within either the upper or lower band.  This routine returns a \tt{SymBandMatrixView}
which straddles the diagonal with \tt{newnlo} super- and sub-diagonals.

\begin{tmvcode}
sb.Transpose()
sb.Conjugate()
sb.Adjoint()
sb.View()
sb.Real()
sb.Imag()
\end{tmvcode}
\index{SymBandMatrix!methods!Transpose}
\index{SymBandMatrix!methods!Conjugate}
\index{SymBandMatrix!methods!Adjoint}
\index{SymBandMatrix!methods!View}
\index{SymBandMatrix!methods!Real}
\index{SymBandMatrix!methods!Imag}
Note that the imaginary part of a complex hermitian band matrix is
skew-symmetric, so \tt{sb.Imag()} is illegal for a \tt{HermBandMatrix}.
If you need to manipulate the imaginary part of a \tt{HermMatrix}, 
you could use
\tt{sb.UpperBandOff().Imag()} 
(since all the diagonal elements are real).
\vspace{12pt}

\subsubsection{Functions}
\index{SymBandMatrix!functions of}

\begin{tmvcode}
RT sb.Norm1() = Norm1(sb)
RT sb.Norm2() = Norm2(sb) = sb.DoNorm2()
RT sb.NormInf() = NormInf(sb)
RT sb.NormF() = NormF(sb) = sb.Norm() = Norm(sb)
RT sb.NormSq() = NormSq(sb)
RT sb.NormSq(RT scale)
RT sb.MaxAbsElement() = MaxAbsElement(sb)
T sb.Trace() = Trace(sb)
T sb.Det() = Det(sb)
RT sb.LogDet(T* sign) = LodDet(sb)
sinv = sb.Inverse() = Inverse(sb)
bool sb.Singular
RT sb.Condition()
RT sb.DoCondition()
sb.Inverse(Matrix<T>& minv)
sb.Inverse(SymMatrix<T>& sinv)
sb.InverseATA(Matrix<T>& cov)
\end{tmvcode}
\index{SymBandMatrix!functions of!Norm1}
\index{SymBandMatrix!functions of!Norm2}
\index{SymBandMatrix!functions of!NormInf}
\index{SymBandMatrix!functions of!MaxAbsElement}
\index{SymBandMatrix!methods!Norm1}
\index{SymBandMatrix!methods!Norm2}
\index{SymBandMatrix!methods!DoNorm2}
\index{SymBandMatrix!methods!NormInf}
\index{SymBandMatrix!methods!MaxAbsElement}
\index{SymBandMatrix!functions of!Norm}
\index{SymBandMatrix!functions of!NormF}
\index{SymBandMatrix!functions of!NormSq}
\index{SymBandMatrix!functions of!Trace}
\index{SymBandMatrix!functions of!Det}
\index{SymBandMatrix!functions of!LogDet}
\index{SymBandMatrix!functions of!Inverse}
\index{SymBandMatrix!methods!Norm}
\index{SymBandMatrix!methods!NormF}
\index{SymBandMatrix!methods!NormSq}
\index{SymBandMatrix!methods!Trace}
\index{SymBandMatrix!methods!Det}
\index{SymBandMatrix!methods!LogDet}
\index{SymBandMatrix!methods!Singular}
\index{SymBandMatrix!methods!Condition}
\index{SymBandMatrix!methods!DoCondition}
\index{SymBandMatrix!methods!Inverse}
\index{SymBandMatrix!methods!InverseATA}
The inverse of a \tt{SymBandMatrix} is not (in general) banded.  
However, it is symmetric (or hermitian).
So \tt{sb.Inverse()} may be assigned to either a \tt{Matrix} or a \tt{SymMatrix}.

\begin{tmvcode}
sb.Zero()
sb.SetAllTo(T x)
sb.Clip(RT thresh)
sb.SetToIdentity(T x = 1)
sb.ConjugateSelf()
sb.TransposeSelf()
Swap(sb1,sb2)
\end{tmvcode}
\index{SymBandMatrix!methods!Zero}
\index{SymBandMatrix!methods!SetAllTo}
\index{SymBandMatrix!methods!Clip}
\index{SymBandMatrix!methods!SetToIdentity}
\index{SymBandMatrix!methods!ConjugateSelf}
\index{SymBandMatrix!methods!TransposeSelf}
\index{SymBandMatrix!functions of!Swap}
\vspace{12pt}

\subsubsection{Arithmetic}
\index{SymBandMatrix!arithmetic}

In addition to \tt{x}, \tt{v}, \tt{m}, \tt{b} and \tt{s} from before, 
we now add \tt{sb} for a \tt{SymBandMatrix}.

\begin{tmvcode}
sb2 = -sb1
sb2 = x * sb1
sb2 = sb1 [*/] x
sb3 = sb1 [+-] sb2
m2 = m1 [+-] sb
m2 = sb [+-] m1
b2 = b1 [+-] sb
b2 = sb [+-] b1
s2 = s1 [+-] sb
s2 = sb [+-] s1
sb [*/]= x
sb2 [+-]= sb1
m [+-]= sb
b [+-]= sb
s [+-]= sb
v2 = sb * v1
v2 = v1 * sb
v *= sb
b = sb1 * sb2
m2 = sb * m1
m2 = m1 * sb
m *= sb
b2 = sb * b1
b2 = b1 * sb
b *= sb
m2 = sb * s1
m2 = s1 * sb
sb2 = sb1 [+-] x
sb2 = x [+-] sb1
sb [+-]= x
sb = x
\end{tmvcode}

\subsubsection{Division}
\index{SymBandMatrix!arithmetic!division}

The division operations are:
\begin{tmvcode}
v2 = v1 [/%] sb
m2 = m1 [/%] sb
m2 = sb [/%] m1
m = sb1 [/%] sb2
s = x [/%] sb
v [/%]= sb
m [/%]= sb
\end{tmvcode}
\tt{SymBandMatrix} has three possible choices for the division decomposition:
\begin{enumerate}
\item
\tt{m.DivideUsing(tmv::LU)} actually does the \tt{BandMatrix} version of 
LU, rather than a Bunch-Kaufman algorithm like for \tt{SymMatrix}.  The
reason is that the pivots in the Bunch-Kaufman algorithm can arbitrarily
expand the band width required to hold the information.  The generic
banded LU algorithm is limited to 3*\tt{nlo}+1 bands.

To access this decomposition, use:
\begin{tmvcode}
bool sb.LUD().IsTrans()
tmv::LowerTriMatrix<T,UnitDiag> sb.LUD().GetL()
tmv::ConstBandMatrixView<T> sb.LUD().GetU()
int* sb.LUD().GetP()
\end{tmvcode}
\index{SymBandMatrix!methods!LUD}
\index{SymBandMatrix!LU decomposition}
\index{LU Decomposition!SymBandMatrix}
The following should result in a matrix numerically very close to \tt{sb}.
\begin{tmvcode}
tmv::Matrix<T> m2(sb.nrows(),sb.ncols);
tmv::MatrixView<T> m2v = 
      sb.LUD().IsTrans() ? m2.Transpose() : m2.View();
m2v = sb.LUD().GetL() * sb.LUD().GetU();
m2v.ReversePermuteRows(sb.LUD().GetP());
\end{tmvcode}

\item
\tt{sb.DivideUsing(tmv::CH)} will perform a Cholesky decomposition.  
\tt{sb} must be hermitian (or real symmetric) to use \tt{CH}, since that is the
only kind of matrix that has a Cholesky decomposition.  

As with a regular \tt{SymMatrix},
the only real advantage of Cholesky over LU decomposition is speed.  If you know your 
matrix is positive-definite, the Cholesky decomposition is the fastest way to 
do division.

If \tt{sb} is tri-diagonal (i.e. \tt{nlo} = 1), then we use a slightly 
different algorithm, which avoids the square roots required for a
Cholesky decomposition.  
Namely, we form the decomposition $sb = LDL^\dagger$, where $L$ is a
unit-diagonal lower banded matrix with 1 sub-diagonal, and $D$ is diagonal.

If \tt{sb} has \tt{nlo} $> 1$, then we just use a normal Cholesky algorithm
where $sb = LL^\dagger$ and $L$ is lower banded with the same \tt{nlo} as
\tt{sb}.

Both versions of the algorithm are accessed with the same methods:
\begin{tmvcode}
BandMatrix<T> sb.CHD().GetL()
DiagMatrix<T> sb.CHD().GetD()
\end{tmvcode}
\index{SymBandMatrix!methods!CHD}
\index{SymBandMatrix!Cholesky decomposition}
\index{Cholesky Decomposition!SymBandMatrix}
with $L$ being made unit-diagonal or $D$ being set to the identity matrix
as appropriate.  (Obviously, GetL() contains all of the information for the non-tridiagonal
version.)

The following should result in a matrix numerically very close to \tt{sb}.
\begin{tmvcode}
Matrix<T> m2 = sb.CHD().GetL() * sb.CHD().GetD() * 
      sb.CHD().GetL().Adjoint()
\end{tmvcode}

\item
\tt{sb.DivideUsing(tmv::SV)} will perform either an eigenvalue decomposition
(for hermitian band and real symmetric band matrices) or a regular singular value
decomposition (for complex symmetric band matrices).

To access this decomposition, use:
\begin{tmvcode}
ConstMatrixView<T> sb.SVD().GetU()
DiagMatrix<RT> sb.SVD().GetS()
Matrix<T> sb.SVD().GetV()
\end{tmvcode}
\index{SymBandMatrix!methods!SVD}
\index{SymBandMatrix!singular value decomposition}
\index{Singular Value Decomposition!SymBandMatrix}
(As for \tt{SymMatrix}, a complex symmetric matrix needs to use the accessor
\tt{SymSVD()} instead, whose \tt{GetS} and \tt{GetV} methods return Views
rather than instantiated matrices.)

The following should result in a matrix numerically very close to \tt{sb}.
\begin{tmvcode}
Matrix<T> m2 = sb.SVD().GetU() * sb.SVD().GetS() * sb.SVD().GetV()
\end{tmvcode}

Both versions also have the same control and access routines as a regular SVD:
(See \ref{matrixsvd}),
\begin{tmvcode}
sb.SVD().Thresh(RT thresh)
sb.SVD().Top(int nsing)
RT sb.SVD().Condition()
int sb.SVD().GetKMax()
\end{tmvcode}
(Likewise for \tt{sb.SymSVD()}.)

\end{enumerate}
The routines 
\begin{tmvcode}
sb.SaveDiv()
sb.SetDiv()
sb.ReSetDiv()
sb.UnSetDiv()
sb.DivideInPlace()
\end{tmvcode}
\index{SymBandMatrix!methods!SaveDiv}
\index{SymBandMatrix!methods!SetDiv}
\index{SymBandMatrix!methods!ReSetDiv}
\index{SymBandMatrix!methods!UnSetDiv}
\index{SymBandMatrix!methods!DivIsSet}
\index{SymBandMatrix!methods!DivideInPlace}
work the same as for regular \tt{Matrix}es.
(See \ref{matrixsetdiv}.)

\subsubsection{Input/Output}
\index{SymBandMatrix!input/output}

The simplest output is the usual:
\begin{tmvcode}
os << sb
\end{tmvcode}
where \tt{os} is any \tt{std::ostream}.
The output format is the same as for a \tt{Matrix}.
(See \ref{matrixio}.)

There is also a compact format for a \tt{BandMatrix}:
\begin{tmvcode}
sb.WriteCompact(os)
\index{SymMatrix!methods!WriteCompact}
\end{tmvcode}
outputs in the format:
\begin{tmvcode}
hB/sB n nlo
( sb(0,0)  )
( sb(1,0)  sb(1,1)  )
...
( sb(nlo,0)  sb(nlo,1) ...  sb(nlo,nlo) )
( sb(nlo+1,1)  sb(nlo+1,2) ...  sb(nlo+1,nlo+1) )
...
( sb(n-1,n-nlo-1)  ... sb(n-1,n-1) )
\end{tmvcode}
where \tt{hB/sB} means \underline{either} \tt{hB} or \tt{sB}, 
which indicates whether the matrix is hermitian or symmetric.

The same compact format can be read back in the usual two ways:
\begin{tmvcode}
tmv::SymBandMatrix<T> sb(n,nlo);
tmv::HermBandMatrix<T> hb(n,nlo);
is >> sb >> hb;
std::auto_ptr<tmv::SymBandMatrix<T> > psb;
std::auto_ptr<tmv::HermBandMatrix<T> > phb;
is >> psb >> phb;
\end{tmvcode}

One can write small values as 0 with
\begin{tmvcode}
sb.Write(std::ostream& os, RT thresh)
sb.WriteCompact(std::ostream& os, RT thresh)
\end{tmvcode}
\index{SymMatrix!methods!WriteCompact}
\index{SymMatrix!methods!Write}

\newpage
\section{Errors and Exceptions}
\index{Exceptions}
\label{error}

There are two kinds of errors that the TMV library looks for.  The first are 
coding errors.  Some examples are:
\begin{itemize}
\item
Trying to access elements outside the range of a \tt{Vector} or \tt{Matrix}.
\item
Trying to add to \tt{Vector}s or \tt{Matrix}es that are different sizes.
\item
Trying to multiply a \tt{Matrix} by a \tt{Vector} where the number of columns 
in the \tt{Matrix} doesn't match the size of the \tt{Vector}.
\item
Viewing a \tt{Matrix} as a \tt{HermMatrix} when the diagonal isn't real.
\item
Calling \tt{m.LUD()}, \tt{m.QRD()}, etc. for a \tt{Matrix} that does not have that 
decomposition set (and saved).
\end{itemize}

I check for all of these (and similar) errors using assert statements.
If these asserts fail, it should mean that
the programmer made a mistake in the code.  (Unless I've made a 
mistake in the TMV code, of course.)

Once the code is working, you can make the code slightly faster by 
compiling with -DNDEBUG.  I say slightly, since most of these checks 
are pretty innocuous.  And most of the computing time is usually in the depths
of the various algorithms, not in these $O(1)$ time checks of the dimensions and such.  
There are a few checks which take $O(N)$ time, but these are only done
if the code is compiled with the \tt{-DXTEST} flag.

The other kind of error checked for by the code is 
where the data don't behave in the way
the programmer expected.  Here is a (complete) list of these errors:
\begin{itemize}
\item
A singular matrix is encountered in a division routine that cannot handle it.
\item
An input file has the wrong format.
\item
A Cholesky decomposition is attempted for a
hermitian matrix that isn't positive definite.
\end{itemize}

These errors are always checked for even if -DNDEBUG is used.
That's because they are not problems in the code per se, but rather
are problems with the data or files used by the code.  So they could
still happen even after the code has been thoroughly tested.

All errors in the TMV library are indicated by throwing an object of type
\tt{tmv::Error}.  If you decide to catch it, you can determine what went
wrong by printing it:
\begin{tmvcode}
catch (tmv::Error& e) {
  std::cerr << e << std::endl;
}
\end{tmvcode}
If you catch the error by value (i.e. without the \tt{\&}), it will print out
a single line description.  If you catch it by reference (as above), it may 
print out more information about the problem.

Also, \tt{tmv::Error} derives from \tt{std::exception} and overrides the \tt{what()}
method, so any program that catches these will catch \tt{tmv::Error} as well.

If you want to be more specific, there are a number of classes that derive from
\tt{Error}:

\subsection{FailedAssert}
\index{Exceptions!FailedAssert}
\tt{tmv::FailedAssert} indicates that one of the assert statements failed.
Since these are coding errors,
if you catch this one, you'll probably just want to print out the error and abort
the program so you can fix the bug.  In addition to printing the text of the
assert statement that failed (if you catch by reference), it will also indicate
the file and line number as normal assert macros do.  Unfortunately, it gives the 
line number in the TMV code, rather than in your own code, but hopefully seeing
which function in TMV found the problem will help you figure out which line
in your own code was incorrect.

If you are absolutely sure that the assert failed due to a bug in the TMV code
rather than your own code, please let me know at \myemail.

\subsection{Singular}
\index{Exceptions!Singular}
\tt{tmv::Singular} indicates that you tried to invert or divide by a matrix
that is (numerically) singular.  This may be useful to catch specifically,
since you may want to do something different when you encounter a singular
matrix.  Note however, that this only detects \underline{exactly} singular
matrices.  If a matrix is numerically close to singular, but no actual 
zeros are found, then no error will be thrown, and your results will just be 
unreliable.

\subsection{ReadError}
\index{Exceptions!ReadError}
\tt{tmv::ReadError} indicates that there was some problem reading in a matrix or 
vector from an \tt{istream} input.  If you catch this by reference and write it,
it will give you a fairly specific description of what the problem was as well as
writing the part of the matrix or vector that was read in successfully.

\subsection{NonPosDef}
\index{Exceptions!NonPosDef}
\index{Cholesky Decomposition!NonPosDef exception}
\label{nonposdef}
\tt{tmv::NonPosDef} indicates that you tried to do some operation that requires
a matrix to be positive definite, and it turned out not to be positive definite.
The most common example would be performing a Cholesky decomposition on a 
hermitian matrix.  I suspect that this is the 
most useful exception to catch specifically, as opposed to just via \tt{tmv::Error} base class.

For example, the fastest algorithm for determining whether a matrix is 
(at least numerically) positive definite is to try the Cholesky decomposition
and catch this exception.  To wit:
\begin{tmvcode}
bool IsPosDef(const tmv::GenSymMatrix<T>& m)
{
    assert(m.isherm());
    try {
        HermMatrix<T> m2 = m;
        CH_Decompose(m2.View());
    }    
    catch (tmv::NonPosDef) {
        return false;
    }
    return true;
}
\end{tmvcode}
Or you might want to use Cholesky for division when possible and Bunch-Kaufman otherwise:
\begin{tmvcode}
try {
    m.DivideUsing(tmv::CH);
    m.SetDiv();
}
catch (tmv::NonPosDef) {
    m.DivideUsing(tmv::LU);
    m.SetDiv();
}
x = b/m;
\end{tmvcode}
Note, however, that the speed difference between the two algorithms is only about
20\% or so for typical matrices.  So if a significant fraction of your matrices are 
not positive definite, you are probably better off always using the LU algorithm.  
Code like that given above would probably be most useful when all of your 
matrices \underline{should} be positive definite in exact arithmetic, but you 
want to guard against one failing the Cholesky decomposition due to round-off
errors.

\index{QR Decomposition!Downdate!NonPosDef exception}
It is also worth mentioning that the routine \tt{QR\_Downdate} described in \S\ref{downdate}
below will also throw the exception \tt{NonPosDef} when it fails.

\subsection{Warnings}
\index{Warnings}
\label{warnings}

There are also a few things that are not really errors, but where the user might be doing
something wrong, or where something unexpected happened, but TMV was able to handle it.
Here is a complete list of these situations:
\begin{itemize}
\item The function \tt{InverseATA()} was called for a matrix with more columns than rows.
The result in this case is really $(AA^\dagger)^{-1}$ instead of $(A^\dagger A)^{-1}$.
\index{Matrix!methods!InverseATA}
\index{BandMatrix!methods!InverseATA}
\index{SymMatrix!methods!InverseATA}
\index{SymBandMatrix!methods!InverseATA}
\item The function \tt{Norm2()} or \tt{Condition()} was called without the SV decomposition
being previously calculated (or saved) for the matrix.  This is often inefficient, but if it is really what you
want, then \tt{DoNorm2()} or \tt{DoCondition()} will bypass the warning.
\index{Matrix!methods!Norm2}
\index{UpperTriMatrix!methods!Norm2}
\index{BandMatrix!methods!Norm2}
\index{SymMatrix!methods!Norm2}
\index{SymBandMatrix!methods!Norm2}
\index{Matrix!methods!Condition}
\index{UpperTriMatrix!methods!Condition}
\index{BandMatrix!methods!Condition}
\index{SymMatrix!methods!Condition}
\index{SymBandMatrix!methods!Condition}
\item The LAPACK function \tt{dstegr} (or \tt{sstegr}) had an error, and the function
\tt{dstedc} (or \tt{sstedc}) was called instead.  The \tt{dstegr} calculation was wasted,
although I don't think it is knowable a priori that this might happen.
\index{LAPACK!dstegr warnings}
\item A LAPACK function requested more workspace than was provided.  This shouldn't happen
anymore, since I switched to using the LAPACK workspace queries.  But the code to check
for this is still there.
\index{LAPACK!workspace warnings}
\end{itemize}

The default way of handling these situations is to print a message to \tt{stdout}.  This is often
appropriate, since it is probably something the programmer wants to know about.
The first two items, in fact, are probably coding errors.

However, this behavior is clearly not always acceptable.  So we provide a function to 
turn off the warnings:
\begin{tmvcode}
void tmv::NoWarnings();
\end{tmvcode}
\index{Warnings!turn off}

If you want to keep the warnings active, but don't like them being written to \tt{stdout}, then 
you can use:
\begin{tmvcode}
std::ostream* tmv::WriteWarningsTo(std::ostream* newos)
\end{tmvcode}
\index{Warnings!redirect to a file}
This lets you redirect the warnings to some other \tt{ostream}.  It could be some log file
that you look at later.  Or it could even be a \tt{stringstream} so you can examine the text
of the warnings within your code and take appropriate action.

The function \tt{WriteWarningsTo} returns a pointer to the old warning stream
in case you only want to change the warning output stream temporarily.
Also if you give \tt{WriteWarningsTo} a null pointer, rather than an actual \tt{ostream},
then this will turn off the warnings.  In fact the above \tt{NoWarnings} function is 
equivalent to \tt{(void) WriteWarningsTo(0)}.  These two features in combination
let you turn off the warnings temporarily.  For example:
\begin{tmvcode}
tmv::Matrix<double> m(5,20);
// m = ....
tmv::Matrix<double> inv_mmt(5,5);
std::ostream* tempos = tmv::WriteWarningsTo(0);
m.InverseATA(int_mmt); // actually calculates (m*mt)^-1
tmv::WriteWarnignsTo(tempos);
\end{tmvcode}

\newpage
\section{Advanced Usage}

\subsection{Eigenvalues and eigenvectors}
\index{Eigenvalues}

The eigenvalues of a matrix are important quantities in many matrix applications.
A number, $\lambda$, is an eigenvalue of a square matrix, $A$, if for some
non-zero vector $v$,
\begin{equation*}
A v = \lambda v
\end{equation*}
in which case $v$ is the called an eigenvector corresponding to the eigenvalue $\lambda$.
Since any arbitrary multiple of $v$ also satisfies this equation, it is common practice
to scale the eigenvectors so that $||v||_2 = 1$.
If $v_1$ and $v_2$ are eigenvectors whose eigenvalues are 
$\lambda_1 \neq \lambda_2$, then $v_1$ and $v_2$ are linearly independent.

The above equation implies that 
\begin{align*}
A v - \lambda v &= 0 \\
(A - \lambda I) v &= 0 \\
\det(A-\lambda I) &= 0 \quad (\text{or} ~~ v = 0)
\end{align*}
If $A$ is an $N \times N$ matrix, then the last expression is called the characteristic
equation of $A$ and the left hand side is a polynomial of degree $N$.
Thus, it has potentially $N$ solutions.  
Note that, even for real matrices, the solution may yield complex eigenvalues, in which
case, the corresponding eigenvectors will also be complex.

If there are solutions to the characteristic equation which are multiple roots, then these
eigenvalues are said to have a multiplicity greater than $1$.  These eigenvalues 
may have multiple corresponding eigenvectors.  That is, different values of $v$ 
(which are not just a multiple of each other) may satisfy the equation $A v = \lambda v$.

The number of independent eigenvectors
corresponding to an eigenvalue with multiplicity $> 1$ may be less than that multiplicity\footnote{
The multiplicity of the eigenvalue is generally referred to as its algebraic multiplicity.
The number of corresponding eigenvectors is referred to as its geometric multiplicity.
So $1 \leq \text{geometric multiplicity} \leq \text{algebraic multiplicity}$.}.  Such eigenvalues
are called ``defective'', and any matrix with defective eigenvalues is likewise
called defective.

If $0$ is an eigenvalue, then the matrix $A$ is singular.
And conversely, singular matrices necessarily have $0$ as one of their eigenvalues.

If we define $\Lambda$ to be a diagonal matrix with the values of $\lambda$ along
the diagonal, then we have (for non-defective matrices)
\begin{equation*}
A V = V \Lambda
\end{equation*}
where the columns of $V$ are the eigenvectors.  If $A$ is defective, we can construct
a $V$ that satisfies this equation too, but some of the columns will have to be all zeros.
There will be one such column for each missing eigenvector, and the other columns will
be the eigenvectors.

If $A$ is not defective, then all of the columns of $V$ are linearly independent, which
implies that $V$ is not singular (i.e. $V$ is ``invertible'').  Then,
\begin{align*}
A &= V \Lambda V^{-1}\\
\Lambda &= V^{-1} A V
\end{align*}
This is known as ``diagonalizing'' the matrix $A$.  The determinant and trace are preserved by this procedure, which implies two more properties of eigenvalues:
\begin{align*}
\det(A) &= \prod_{k=1}^{N} \lambda_k\\
\text{tr}(A) &= \sum_{k=1}^{N} \lambda_k
\end{align*}

If $A$ is a ``normal'' matrix -- which means that $A$ commutes with its adjoint,
$AA^\dagger = A^\dagger A$ -- then the 
matrix $V$ is unitary, and $A$ cannot be defective.  
The most common example of a normal matrix is a hermitian matrix
(where $A^\dagger = A$), which has
the additional property that all of the eigenvalues are real\footnote{
Other examples of normal matrices are unitary matrices ($A^\dagger A = AA^\dagger = I$)
and skew-hermitian matrices ($A^\dagger = -A$).  However, normal matrices do
not have to be one of these special types.}.

So far, the TMV library can only find the eigenvalues and eigenvectors 
of hermitian matrices.  The routines to do so are 
\begin{tmvcode}
void Eigen(const GenSymMatrix<T>& A,
      const MatrixView<T>& V, const VectorView<RT>& lambda);

void Eigen(const GenSymBandMatrix<T>& A,
      const MatrixView<T>& V, const VectorView<RT>& lambda);
\end{tmvcode}
\index{SymMatrix!eigenvalues and eigenvectors}
\index{SymBandMatrix!eigenvalues and eigenvectors}
\index{Eigenvalues!SymMatrix}
\index{Eigenvalues!SymBandMatrix}
where \tt{V.col(i)} is the eigenvector corresponding to each eigenvalue \tt{lambda(i)}.
The original matrix \tt{A} can be obtained from
\begin{tmvcode}
A = V * DiagMatrixViewOf(lambda) * V.Adjoint();
\end{tmvcode}

There are also routines which only find the eigenvalues, which are faster, since they
do not perform the calculations to determine the eigenvectors:
\begin{tmvcode}
void Eigen(const SymMatrixView<T>& A, const VectorView<RT>& lambda);

void Eigen(const GenSymBandMatrix<T>& A, const VectorView<RT>& lambda);
\end{tmvcode}
\index{SymMatrix!eigenvalues and eigenvectors}
\index{SymBandMatrix!eigenvalues and eigenvectors}
Note that the first one uses the input matrix \tt{A} as workspace and destroys the 
input matrix in the process.

\subsection{Matrix decompositions}
\label{Decompositions}

While many matrix decompositions are primarily useful for performing matrix division
(or least-squares pseudo-division), one sometimes wants to perform the decompositions for 
their own sake.  It is possible to get at the underlying decomposition with the various
divider accessor routines like \tt{m.LUD()}, \tt{m.QRD()}, etc.  However, this is somewhat
roundabout, and at times inefficient.  So we provide direct ways to perform all of
the various matrix decompositions that are implemented by the TMV code.

In the routines below, the matrix being decomposed is input as \tt{A}, and we
list the routines for all of the 
allowed types for \tt{A} for each kind of decomposition.  If \tt{A} is listed as a ``\tt{Gen}''
type, such as \tt{GenBandMatrix<T>}, then that means the input matrix is not 
changed by the decomposition routine.  
If \tt{A} is listed as a ``\tt{View}'' type,
such as \tt{BandMatrixView<T>}, then that means the input matrix is 
changed.  

In some cases where \tt{A} is a \tt{View} type, one of the decomposition components is 
returned in the location of 
\tt{A}, or some part of it, overwriting the input matrix.  
In these cases, there will be a line indicating this
after the function (e.g. \tt{L = A.LowerTri()}).
In other cases, the input matrix is just used as workspace, and it is junk
on output, (in which case, there is no such line following the function).

Sometimes, only certain parts of a decomposition are wanted.  For example,
you might want to know the singular values of a matrix, but not care about
the $U$ and $V$ matrices.  For cases such as this, there are versions
of the decomposition routines which omit certain output parameters.
These routines are generally faster than the versions which include all
output parameters, since they can omit some of the calculations.

Finally, a word about the permutations.  In TMV, permutations are
defined as a series of row or column swaps.  I haven't made a \tt{Permutation}
class yet to make it easy to use these permutations.  But the 
code snippets which show how to recreate the input matrices from
the decompositions should be sufficient to describe how to use the 
permutations as given.

\begin{itemize}

\item LU Decomposition 
\index{LU Decomposition}

(\tt{Matrix}, \tt{BandMatrix})

$A \rightarrow P L U$ where $L$ is lower triangular, 
$U$ is upper triangular, and $P$ is a permutation.

\begin{tmvcode}
void LU_Decompose(const MatrixView<T>& A, int* P);
L = A.LowerTri(UnitDiag);
U = A.UpperTri(NonUnitDiag);

void LU_Decompose(const GenBandMatrix<T>& A, 
      const LowerTriMatrixView<T>& L, 
      const BandMatrixView<T>& U, int* P);
\end{tmvcode}
\index{LU Decomposition!Matrix}
\index{LU Decomposition!BandMatrix}
\index{Matrix!LU decomposition}
\index{BandMatrix!LU decomposition}
In the second case, \tt{U} must have \tt{U.nhi() = A.nlo()+A.nhi()},
and \tt{L} should be \tt{UnitDiag}.
In both cases, \tt{P} must have \tt{A.ncols()} elements of memory allocated.

The original matrix \tt{A} can be obtained from:
\begin{tmvcode}
A = L * U;
A.ReversePermuteRows(P);
\end{tmvcode}

\item Cholesky Decomposition 
\index{Cholesky Decomposition}

(\tt{HermMatrix}, \tt{HermBandMatrix})

$A \rightarrow L L^\dagger$, where $L$ is lower triangular,
and $A$ is hermitian.

\begin{tmvcode}
void CH_Decompose(const SymMatrixView<T>& A);
L = A.LowerTri();

void CH_Decompose(const SymBandMatrixView<T>& A);
L = A.LowerBand();
\end{tmvcode}
\index{Cholesky Decomposition!SymMatrix}
\index{Cholesky Decomposition!SymBandMatrix}
\index{SymMatrix!Cholesky decomposition}
\index{SymBandMatrix!Cholesky decomposition}

The original matrix \tt{A} is very simply
\begin{tmvcode}
A = L * L.Adjoint();
\end{tmvcode}

\item Bunch-Kaufman Decomposition 
\index{Bunch-Kaufman Decomposition}

(\tt{HermMatrix}, \tt{SymMatrix})

If $A$ is hermitian, $A \rightarrow P L D L^\dagger P^T$,
and if $A$ is symmetric, $A \rightarrow P L D L^T P^T$, where $P$ is a permutation,
$L$ is lower triangular, and $D$ is hermitian or symmetric tridiagonal (respectively).  
In fact, $D$ is even more special than that: it is block diagonal with $1 \times 1$
and $2 \times 2$ blocks,
which means that there are no two consecutive
non-zero elements along the off-diagonal.

\begin{tmvcode}
void LDL_Decompose(const SymMatrixView<T>& A, 
      const SymBandMatrixView<T>& D, int* P);
L = A.LowerTri(UnitDiag);
\end{tmvcode}
\index{Bunch-Kaufman Decomposition!SymMatrix}
\index{SymMatrix!Bunch-Kaufman decomposition}
\tt{P} must have \tt{A.size()} elements of memory allocated.

The original matrix \tt{A} can be obtained from:
\begin{tmvcode}
A = L * D * (A.isherm() ? L.Adjoint() : L.Transpose());
A.ReversePermuteRows(P);
A.ReversePermuteCols(P);
\end{tmvcode}

\item Tridiagonal LDL$^\dagger$ Decomposition 
\index{Tridiagonal LDL Decomposition}

(\tt{HermBandMatrix}, \tt{SymBandMatrix} with \tt{nlo = 1})

This decomposition for symmetric or hermitian tri-diagonal matrices is 
similar to the Bunch-Kaufman decomposition: 
$A \rightarrow L D L^\dagger$
or $A \rightarrow L D L^T$ where this time $D$ is a regular diagonal matrix and $L$ is
a lower band matrix with a single subdiagonal and all 1's on the diagonal.

It turns out
that the Bunch-Kaufman algorithm on banded matrices tends to expand the band structure
without limit because of the pivoting involved, so it is not practical.
However, with tridiagonal matrices, it is often possible to perform the 
decomposition without pivoting.  Thus, this one does not have any growth of the band structure,
but it is not as stable for singular or nearly singular matrices.
If an exact zero is found on the diagonal along the way
tmv::NonPosDef is thrown.\footnote{
Note, however, that if $A$ is complex, symmetric - i.e. not hermitian -
then this doesn't actually mean that $A$ is not positive definite (since such a 
quality is only defined for hermitian matrices).  Furthermore, 
hermitian matrices that are not positive definite will probably be decomposed successfully
without throwing, resulting in D having negative values.

Also, the LAPACK implementation gives an error for matrices that the native code
successfully decomposes.  It throws for hermitian matrices whenever they are 
not positive definite, whereas the native code succeeds for many indefinite matrices.
}
\index{Exceptions!NonPosDef}
\index{LAPACK!NonPosDef exceptions from LDL\_Decompose}

\begin{tmvcode}
void LDL_Decompose(const SymBandMatrixView<T>& A);
(L = A.LowerBand()).diag().SetAllTo(T(1));
D = DiagMatrixViewOf(A.diag());
\end{tmvcode}
\index{Tridiagonal LDL Decomposition!SymBandMatrix}
\index{SymBandMatrix!tri-diagonal LDL decomposition}

The original matrix \tt{A} can be obtained from:
\begin{tmvcode}
A = L * D * (A.isherm() ? L.Adjoint() : L.Transpose());
\end{tmvcode}

\item QR Decomposition 
\index{QR Decomposition}

(\tt{Matrix}, \tt{BandMatrix})

$A \rightarrow Q R$, where $Q$ is column-unitary 
(i.e. $Q^\dagger Q = I$), $R$ is upper triangular, and $A$ is either square or 
has more rows than columns.

\begin{tmvcode}
void QR_Decompose(const MatrixView<T>& A, 
      const UpperTriMatrixView<T>& R);
Q = A;

void QR_Decompose(const GenBandMatrix<T>& A, 
      const MatrixView<T>& Q, const BandMatrixView<T>& R);
\end{tmvcode}
\index{QR Decomposition!Matrix}
\index{Matrix!QR decomposition}
\index{QR Decomposition!BandMatrix}
\index{BandMatrix!QR decomposition}
In the second case, \tt{R} must have \tt{R.nhi()} $>=$ \tt{A.nlo()+A.nhi()}.

If you only need $R$, the following versions are faster, since they do 
not fully calculate $Q$.
\begin{tmvcode}
void QR_Decompose(const MatrixView<T>& A);
R = A.UpperTri();

void QR_Decompose(const GenBandMatrix<T>& A, 
      const BandMatrixView<T>& R);
\end{tmvcode}

\item QRP Decomposition 
\index{QRP Decomposition}

(\tt{Matrix})

$A \rightarrow Q R P$, where $Q$ is column-unitary 
(i.e. $Q^\dagger Q = I$), $R$ is upper triangular, $P$ is a permutation, 
and $A$ is either square or has more rows than columns.

\begin{tmvcode}
void QRP_Decompose(const MatrixView<T>& A, 
      const UpperTriMatrixView<T>& R, int* P, bool strict=false);
Q = A;
\end{tmvcode}
\index{QRP Decomposition!Matrix}
\index{Matrix!QRP decomposition}
\tt{P} must have \tt{A.ncols()} elements of memory allocated.

As discussed in \S\ref{divtypes}, 
there are two slightly different algorithms for doing a QRP decomposition.  
If \tt{strict = true}, then the diagonal elements
of $R$ be strictly decreasing (in absolute value) from upper-left to lower-right\footnote{
\index{QRP Decomposition!LAPACK ?geqp3}
\index{LAPACK!Problems with QRP decomposition}
If you are using a LAPACK library, you might find that the output $R$ diagonal
is not always strictly decreasing, although it will usually be close.  If strictly monotonic
diagonal elements are important for you, you can use the native TMV algorithm instead
by compiling with the flag \texttt{-DNOGEQP3}.}.

If \tt{strict} is false, however (the default), then the diagonal elements of $R$ will not
be strictly decreasing.  Rather, there will be no diagonal element
of $R$ below and to the right of one which is more than a factor of $\epsilon^{1/4}$ 
smaller in absolute value, where 
$\epsilon$ is the machine precision.  This restriction
is almost always sufficient to make the decomposition useful for singular or nearly
singular matrices, and it is much faster than the strict algorithm.

The original matrix \tt{A} is obtained from:
\begin{tmvcode}
A = Q * R;
A.ReversePermuteCols(P);
\end{tmvcode}

If you only need $R$, the following versions is faster, since it does
not fully calculate $Q$.
\begin{tmvcode}
void QRP_Decompose(const MatrixView<T>& A, bool strict=false);
R = A.UpperTri();
\end{tmvcode}

\item Singular Value Decomposition 
\index{Singular Value Decomposition}

(\tt{Matrix}, \tt{SymMatrix}, \tt{HermMatrix}, \tt{BandMatrix}, \tt{SymBandMatrix}, \tt{HermBandMatrix})

$A \rightarrow U S V$,
where $U$ is column-unitary (i.e. $U^\dagger U = I$),
$S$ is real diagonal, $V$ is square unitary, and $A$ is either square or 
has more rows than columns.\footnote{
The SVD is more commonly written as $A \rightarrow U S V^T$.  As far as I
can tell, this seems to be a holdover from the days of Fortran programming.  In Fortran, matrices
are stored in column-major format.  Considering that the rows of what we call $V$ are the 
singular vectors, also known as principal components, of $A$, it made more sense for 
Fortran programmers to use the transpose of $V$ which has the principal components in 
the columns.  This complication is unnecessary in TMV.  If you want the principal components
stored contiguously, just make $V$ row-major.  On the other hand, decomposition
with column-major storage of $V$
is usually a bit faster, so you need to make a choice appropriate for your particular program.}

\begin{tmvcode}
void SV_Decompose(const MatrixView<T>& A, 
      const DiagMatrixView<RT>& S, const MatrixView<T>& V);
U = A;

void SV_Decompose(const GenSymMatrix<T>& A, 
      const MatrixView<T>& U, const DiagMatrixView<RT>& S, 
      const MatrixView<T>& V);

void SV_Decompose(const GenBandMatrix<T>& A,
      const MatrixView<T>& U, const DiagMatrixView<RT>& S,
      const MatrixView<T>& V);

void SV_Decompose(const GenSymBandMatrix<T>& A,
      const MatrixView<T>& U, const DiagMatrixView<RT>& S, 
      const MatrixView<T>& V);
\end{tmvcode}
\index{Singular Value Decomposition!Matrix}
\index{Singular Value Decomposition!BandMatrix}
\index{Singular Value Decomposition!SymMatrix}
\index{Singular Value Decomposition!SymBandMatrix}
\index{Matrix!singular value decomposition}
\index{BandMatrix!singular value decomposition}
\index{SymMatrix!singular value decomposition}
\index{SymBandMatrix!singular value decomposition}

The input $A$ matrix must not have more columns than rows.  If you want
to calculate the SVD of such a matrix, you should decompose $A^T$ instead:
\begin{tmvcode}
tmv::Matrix<double> A(nrows,ncols); // ncols > nrows
[ A = ... ]
tmv::Matrix<double> V = A;
tmv::DiagMatrix<double> S(nrows);
tmv::Matrix<double> U(nrows,nrows);
SV_Decompose(V.Transpose(),S.View(),U.Transpose());
// Now A = U * S * V
\end{tmvcode}

If you only need $S$, or $S$ and $V$, or $S$ and $U$, the following 
versions are faster, since they do
not fully calculate the omitted matrices.  
\begin{tmvcode}
void SV_Decompose(const MatrixView<T>& A, 
      const DiagMatrixView<RT>& S, const MatrixView<T>& V, false);
// U != A

void SV_Decompose(const MatrixView<T>& A, 
      const DiagMatrixView<RT>& S, bool StoreU);
if (StoreU) U = A;

void SV_Decompose(const SymMatrixView<T>& A, 
      const DiagMatrixView<RT>& S);
      
void SV_Decompose(const GenSymMatrix<T>& A,
      const DiagMatrixView<RT>& S, const MatrixView<T>& V);
      
void SV_Decompose(const GenSymMatrix<T>& A,
      const MatrixView<T>& U, const DiagMatrixView<RT>& S);
      
void SV_Decompose(const GenBandMatrix<T>& A, 
      const DiagMatrixView<RT>& S);

void SV_Decompose(const GenBandMatrix<T>& A,
      const DiagMatrixView<RT>& S, const MatrixView<T>& V);
      
void SV_Decompose(const GenBandMatrix<T>& A,
      const MatrixView<T>& U, const DiagMatrixView<RT>& S);
      
void SV_Decompose(const GenSymBandMatrix<T>& A, 
      const DiagMatrixView<RT>& S);
      
void SV_Decompose(const GenSymBandMatrix<T>& A,
      const DiagMatrixView<RT>& S, const MatrixView<T>& V);

void SV_Decompose(const GenSymBandMatrix<T>& A,
      const MatrixView<T>& U, const DiagMatrixView<RT>& S);
\end{tmvcode}

\item Polar Decomposition 
\index{Polar Decomposition}

(\tt{Matrix}, \tt{BandMatrix})

$A \rightarrow U P$ where $U$ is unitary and $P$ is positive definite hermitian.

This is similar to polar form of a complex number: $z = r e^{i \theta}$.
In the matrix version, $P$ acts as $r$, being in some sense the ``magnitude'' 
of the matrix.  And $U$ acts as $e^{i \theta}$, being a generalized rotation.

\begin{tmvcode}
void Polar_Decompose(const MatrixView<T>& A, 
      const SymMatrixView<T>& P);
U = A;

void Polar_Decompose(const GenBandMatrix<T>& A,
      const MatrixView<T>& U, const SymMatrixView<T>& P);
\end{tmvcode}
\index{Polar Decomposition!Matrix}
\index{Polar Decomposition!BandMatrix}
\index{Matrix!polar decomposition}
\index{BandMatrix!polar decomposition}

\item Matrix Square Root 

(\tt{HermMatrix, HermBandMatrix})

$A \rightarrow S S$, where $A$ and $S$ are each positive definite hermitian matrices.

\begin{tmvcode}
void SquareRoot(const SymMatrixView<T>& A);
S = A;

void SquareRoot(const GenSymBandMatrix<T>& A, 
      const SymMatrixView<T>& S);
\end{tmvcode}
\index{SymMatrix!square root}
\index{SymBandMatrix!square root}
\index{Square Root!SymMatrix}
\index{Square Root!SymBandMatrix}

If $A$ is found to be not positive definite, a \tt{NonPosDef} exception is thrown.
\index{Exceptions!NonPosDef}

\end{itemize}

\subsection{Update a QR decomposition}
\index{QR Decomposition!Update}

One reason that it can be useful to create and deal with the QR decomposition directly,
rather than 
just relying on the division routines is the possibility
of updating or ``downdating'' the resulting $R$ matrix.  

If you are doing a 
least-square fit to a large number of linear equations, you can write the system as
a matrix equation: 
$A x = b$, where $A$ is a matrix with more rows than columns, and you are seeking,
not an exact solution for $x$, but rather the value of $x$ which minimizes
$||b-Ax||_2$.  See \S\ref{leastsquare} for a more in depth discussion of this topic.

It may be the case that you
have more rows (i.e. constraints) than would allow the entire matrix to fit in memory.  
In this case it may be tempting to use the so-called normal equation instead: 
\begin{align*}
A^\dagger A x &= A^\dagger b \\
x & = (A^\dagger A)^{-1} A^\dagger b
\end{align*}
This equation theoretically gives the same 
solution as using the QR decomposition on the original design matrix.
However, it can be shown that the condition of $A^\dagger A$ is the 
\underline{square} of the condition of $A$.  Since larger condition values
lead to larger numerical instabilities and round-off problems, a mildly
ill-conditioned matrix is made much worse by this procedure.

When all of $A$ fits in memory, the better solution is to use the QR decomposition, $A = QR$,
to calculate $x$.
\begin{align*}
Q R x &= b \\
x &= R^{-1} Q^\dagger b
\end{align*}
In fact, this is the usual behind-the-scenes procedure when you write $x = b/A$ in TMV.
But if $A$ is too large to fit in memory, then so is $Q$.

A compromise solution, which is not quite as good as doing the full QR decomposition,
but is better than using the normal equation, is to just calculate the $R$ of the
QR decomposition, and not $Q$.  Then:
\begin{align*}
A^\dagger A x &= A^\dagger b \\
R^\dagger Q^\dagger Q R x = R^\dagger R x &= A^\dagger b \\
x &= R^{-1} (R^\dagger)^{-1} A^\dagger b
\end{align*}

Calculating $R$ directly from $A$ is numerically much more stable than 
calculating it through, say, a Cholesky decomposition of $A^\dagger A$.
So this method produces a more accurate answer for $x$ than the normal equation does.

But how can $R$ be calculated if we cannot fit all of $A$ into memory at once?

First, we point out a characteristic of unitary matrices that the product 
of two or more of them is also unitary.  
This implies that if we can calculate
something like: $A = Q_0 Q_1 Q_2 ... Q_n R$, then this is the $R$ that we want.

So, consider breaking $A$ into a submatrix, $A_0$, which can fit into memory, 
plus the remainder, $A_1$.
\begin{equation*}
A = \left(\begin{array}{c}A_0 \\A_1\end{array}\right)
\end{equation*}
First perform a QR decomposition of $A_0 = Q_0 R_0$.  Then we have:
\begin{align*}
A &= \left(\begin{array}{c}Q_0 R_0 \\ A_1 \end{array}\right) \\
&= \left(\begin{array}{cc}Q_0 & 0 \\ 0 & 1\end{array}\right) 
      \left(\begin{array}{c}R_0 \\ A_1 \end{array}\right) \\
&\equiv Q_0^\prime A_1^\prime
\end{align*}

Assuming that $A_0$ has more rows than columns, 
then $A_1^\prime$ has fewer rows than
the original matrix $A$.  So we can iterate this process until the 
resulting matrix can fit in memory, and we can perform the final QR update
to get the final value of $R$.

For the numerical reasons mentioned above, 
the fewer such iterations you do, the better.  So you should try to include as many
rows of the matrix $A$ as possible in each step, given the amount of memory
available.

The solution equation, written above, also needs the quantity $A^\dagger b$, which
can be accumulated in the same blocks:
\begin{equation*}
A^\dagger b = A_1^\dagger b_1 + A_2^\dagger b_2 + ....
\end{equation*}
This, combined with the calculation of $R$, allows us to determine $x$ using the above formula.

The TMV library includes a command which does the update step of the above procedure
directly, which is slightly more efficient than explicitly forming the $A_k^\prime$ matrices.
The commands is
\begin{tmvcode}
void QR_Update(const UpperTriMatrixView<T>& R, const MatrixView<T>& M)
\end{tmvcode}
which updates the value of $R$ such that $R_{\rm out}^\dagger R_{\rm out} =
R_{\rm in}^\dagger R_{\rm in} + M^\dagger M$.
(The input matrix \tt{M} is destroyed in the process.)  This is equivalent to the QR
definition of the update described above.

So the entire process might be coded using TMV as:
\begin{tmvcode}
int n_full = nrows_for_full_A_matrix;
int n_mem = nrows_that_fit_in_memory;
assert(n_mem < n_full);
assert(n_mem > ncols);

tmv::Matrix<double> A(n_mem,ncols); 
tmv::Vector<double> b(n_mem);

// Import_Ab sets A to the first n_mem rows of the full matrix, 
// and also sets b to the same components of the full rhs vector.
// Maybe it reads from a file, or performs a calculation, etc.
Import_Ab(0,n_mem,A.View(),b.View());

// x will be the solution to A_full x = b_full when we are done
// But for now, it is accumulating A_full.Transpose() * b_full.
tmv::Vector<double> x = A.Transpose() * b;

// Do the initial QR Decomposition:
QR_Decompose(A.View());
tmv::UpperTriMatrix<double> R = A.UpperTri();

// Iterate until we have done all the rows
int n1 = 0, n2 = n_mem;
while (n2 < n_full) 
{
    n1 = n2; n2 += n_mem;
    if (n2 > n_full) n2 = n_full;
    // (Usually, A1==A, b1==b, but not the last time through the loop.)
    tmv::MatrixView<double> A1 = A.Rows(0,n2-n1);
    tmv::VectorView<double> b1 = b.SubVector(0,n2-n1);

    // Import the next bit:
    Import_Ab(n1,n2,A1,b1);

    // Update, x, R:
    x += A1.Transpose() * b1;
    QR_Update(R.View(),A1);
}

// Finish the solution:
x /= R.Transpose();
x /= R;
\end{tmvcode}

\subsection{Downdate a QR decomposition}
\index{QR Decomposition!Downdate}
\label{downdate}

When performing a least-square fit of some data to a model,
it is common to do some kind of outlier rejection to remove data that
seem not to be applicable to the model - things like spurious measurements
and such.
For this, we basically want the opposite of a QR update - instead we want to 
find the QR decomposition that results from
removing a few rows from $A$.  This is called a QR ``downdate'', and is performed
using the subroutine:
\begin{tmvcode}
void QR_Downdate(const UpperTriMatrixView<T>& R, const GenMatrix<T>& M)
\end{tmvcode}
\tt{M} represents the rows from the original matrix to remove from the 
QR decomposition.

It is possible for the downdate to fail (and throw an exception) 
if the matrix $M$ does not represent rows
of the matrix that was originally used to create $R$.
Furthermore,
with round-off errors, the error may still result with actual rows from the 
original $A$
if $R$ gets too close to singular.  In this case, \tt{QR\_Downdate} throws
a \tt{NonPosDef} exception.  This might seem like a strange choice, but the 
logic is that $R^\dagger R$ is the Cholesky decomposition of $A^\dagger A$,
and \tt{QR\_Downdate(R,M)} basically updates $R$ to be the Cholesky decomposition
of $A^\dagger A - M^\dagger M$.  The procedure fails (and throws) when this latter 
matrix is found not to be positive definite.
\index{QR Decomposition!Downdate!NonPosDef exception}
\index{Exceptions!NonPosDef}

It is worth pointing out that most of the texts and online resources that discuss the 
QR downdate algorithm only explain how to do one row at a time, using a 
modification of the QR update using Givens rotations.  
If you are doing many rows, it is common that roundoff errors in such a 
procedure accumulate sufficiently for the routine to fail.  The TMV algorithm
instead downdates all of the rows together using a modification of the 
Householder reflection algorithm for updates.  This algorithm seems to be
much more stable that the former one.  I have not seen this algorithm discussed
anywhere (although I am sure that I am not the first to come up with it), so
I'll provide a short summary here.

\subsubsection{The update algorithm}
\index{QR Decomposition!Update!algorithm}

First lets look at the Householder algorithm for QR update (since the downdate algorithm
is intimately related):

Given the initial decomposition $A_0 = Q_0 R_0$, we want to find $R$ such that
\begin{align*}
A = \left(\begin{array}{c}A_0 \\ A_1 \end{array}\right) &= Q_1 R \\
\left(\begin{array}{c}Q_0 R_0 \\ A_1 \end{array}\right) &= Q_1 R \\
\left(\begin{array}{cc}Q_0 & 0 \\ 0 & 1 \end{array}\right) 
\left(\begin{array}{c}R_0 \\ A_1 \end{array}\right) &= Q_1 R 
\end{align*}
So if we perform a QR decomposition: 
\begin{equation*}
A_2 \equiv \left(\begin{array}{c}R_0 \\ A_1 \end{array}\right) = Q_2 R 
\end{equation*}
Then this is the $R$ we want, and 
\begin{equation*}
Q_1 = \left(\begin{array}{cc}Q_0 & 0 \\ 0 & 1 \end{array}\right)  Q_2
\end{equation*}

For the following discussion, let $N$ be the number of rows (and columns) in $R_0$,
and $M$ be the number of rows in $A_1$.

To perform the decomposition, we multiply $A_2$ by a series of Householder reflections
on the left to zero out each column of $A_1$ one at a time.  Householder reflections 
are unitary, so their product in the reverse order is $Q_2$:
\begin{align*}
\left(\begin{array}{c}R \\ 0 \end{array}\right) &=
  H_N H_{N-1} ... H_2 H_1 
  \left(\begin{array}{c}R_0 \\ A_1 \end{array}\right)  \\
Q_2 &= H_1^\dagger H_2^\dagger ... H_{N-1}^\dagger H_N^\dagger 
\end{align*}

Householder reflections are defined as $H = I - \beta (x - y e_1)  (x - y e_1)^\dagger$
where $x$ is a (column) vector, $y$ is a scalar with $|y| = ||x||_2$, $e_1$ is the
basis vector whose only non-zero element is the first: $e_1(1) = 1$, and
$\beta = (||x||_2^2 - y^* x(1))^{-1}$.
They have the useful properties that $H x = y e_1$ and they are unitary:
$H^\dagger H = H H^\dagger = I$.
Furthermore, if $\beta$ is real
\footnote{
Unfortunately, LAPACK chose to implement complex Householder matrices 
with $y$ chosen to be real, which means that $\beta$ is complex.  Hence
$H$ is not Hermitian, which complicates a lot of the calling routines which use
Householder matrices.  Since TMV is designed to be able to call LAPACK
when possible, it is forced to follow the same convention.

In fact, it could be argued that their convention is even ``wrong'' in the sense that
their Householder matrices are not actually ``reflections''.  A reflection is a 
unitary matrix whose determinant is $-1$.  The determinant of a Householder 
matrix as defined here is $-\beta^2/|\beta|^2$ which is $-1$ for real $\beta$, 
but not for complex $\beta$.  But we are stuck with their choice, so we allow $\beta$
to be complex in this discussion.
}, they are also hermitian: $H = H^\dagger$.
\index{LAPACK!Householder matrices}

$H_1$ is defined for the vector 
$x = (R_0(1,1), 0, 0, ... , 0, 0, A_1(1,1), A_1(2,1), ... , A_1(M,1) )$ 
where the stretch of $0$'s includes a total of $(N-1)$ $0$'s. This value of 
$x$ completely determines the Householder matrix $H_1$ up to an arbitrary sign
on either $y$ or $\beta$ (or in general an arbitrary factor $e^{i \theta}$) which is 
chosen to minimize rounding errors.

The product $H_1 A_2$ ``reflects'' the first column
of $A_1$ into the first diagonal element of $R_0$.  Because of all the $0$'s, 
most of $R_0$ is unaffected -- only the first row of $R_0$ and the rest of $A_1$
are changed.
The subsequent Householder reflections are defined similarly, each zeroing out
a column of $A_1$, and modifying the corresponding row of $R_0$ and the 
remaining elements of $A_1$.

At the end of this procedure, the matrix $R_0$ will be changed into the 
matrix $R$.  If desired, $Q$ may also be calculated in the process, but the 
TMV implementation of the QR update does not calculate $Q$.

\subsubsection{The downdate algorithm}
\index{QR Decomposition!Downdate!algorithm}

Given the initial decomposition
\begin{equation*}
A = \left(\begin{array}{c}A_0 \\ A_1 \end{array}\right) = Q_1 R 
\end{equation*}
we want to find $R_0$ such that $A_0 = Q_0 R_0$.

The algorithm to do this essentially performs the same steps as in the update
algorithm above,
but instead removes the effect of each $H$ from $R$.
This is easy to
do if we can determine what each $H$ is, since $H^{-1} = H^\dagger$, so we just
apply $H^\dagger$ to update each row of $R$.  The $A_1$ update takes
the regular $H$ matrix, since we need to replicate the steps that we would do
for an update to keep finding the correct values for the remaining columns of $A_1$.

All of the values in the vector $x$ needed to define $H_1$ are given, except for the first,
$R_0(0,0)$.  But this is easy to calculate, since
\begin{equation*}
|R(0,0)|^2 = |R_0(0,0)|^2 + ||A_1(1:M,0)||^2
\end{equation*}
This determines the $x$ vector, which in turn defines $H_1$
(modulo an arbitrary sign, which again is chosen to minimize rounding errors).
Thus, we can calculate $H_1$ and apply it as described above.  Each subsequent Householder
matrix is created and applied similarly for each column of $A_1$.  When we have finished
this process, we are left with $R_0$ in the place of $R$.

If at any point in the process, we find the calculated $|R_0(k,k)|^2 < 0$, then 
the algorithm fails.  In the TMV implementation, a \tt{NonPosDef} exception is thrown.

In practice, for both of these algorithms, we actually use a blocked implementation for updating
the $R$ and $A_1$ matrices.  We accumulate the effect of the Householder matrices until 
there are sufficiently many (e.g. 64), at which point we update the appropriate rows of the $R$
matrix and the rest of $A_1$.  Implementing this correctly is mostly a matter of keeping track
of which elements have been updated yet, making sure that whenever an element is used,
it is already updated, while delaying as much of the calculation as possible in order 
to make maximum
use of the so-called Level 3 BLAS functions, which are the most efficient on modern computers.
We also make the additional improvement of using a recursive algorithm within each block,
which gains some additional Level 3 BLAS calls, for a bit more speed-up.

\subsection{Other SymMatrix operations}
\index{SymMatrix!arithmetic!rank-2 Update}
\index{SymMatrix!arithmetic!rank-2k Update}
\index{SymMatrix!arithmetic!product of two regular matrices}

There are three more arithmetic routines that we provide for \tt{SymMatrix},
which do not have
any corresponding shorthand with the usual arithmetic operators.

The first two are:
\begin{tmvcode}
tmv::Rank2Update<bool add>(T x, const GenVector<T1>& v1, 
      const GenVector<T2>& v2, const SymMatrixView<T>& s)
tmv::Rank2KUpdate<bool add>(T x, const GenMatrix<T1>& m1, 
      const GenMatrix<T2>& m2, const SymMatrixView<T>& s)
\end{tmvcode}
They are similar to the \tt{Rank1Update} and \tt{RankKUpdate} routines,
which are implemented in TMV with the expressions 
\tt{s += x * v \^\ v} and \tt{s += x * m * m.Transpose()}.

A rank-2 update calculates
\begin{tmvcode}
s (+=) x * ((v1 ^ v2) + (v2 ^ v1))
s (+=) x * (v1 ^ v2.Conjugate()) + conj(x) * (v2 ^ v1.Conjugate())
\end{tmvcode}
for a symmetric or hermitian \tt{s} respectively,
where ``(+=)'' means ``+='' if \tt{add} is \tt{true} and ``='' 
if \tt{add} is \tt{false}.
Likewise, a rank-2k update calculates:
\begin{tmvcode}
s (+=) x * (m1 * m2.Transpose() + m2 * m1.Transpose())
s (+=) x * m1 * m2.Adjoint() + conj(x) * m2 * m1.Adjoint()
\end{tmvcode}
for a symmetric or hermitian \tt{s} respectively.

We don't have an arithmetic operator 
shorthand for these, because, as you can see, the operator
overloading required would be quite complicated.  
And since they are pretty rare, I decided to just let the programmer 
call the routines explicitly.

The other routine is:
\begin{tmvcode}
tmv::SymMultMM<bool add>(T x, const GenMatrix<T>& m1, 
      const GenMatrix<T>& m2, const SymMatrixView<T>& s)
\end{tmvcode}
This calculates the usual generalized matrix product:
\tt{s (+=) x * m1 * m2}, but it basically
asserts that the product \tt{m1 * m2} is symmetric (or hermitian as appropriate).

Since a matrix product is not in general symmetric, I decided not to allow 
this operation with just the usual operators to prevent the user from doing 
this accidentally.  However, there are times when the 
programmer can know that the product should be (at least numerically close to)
symmetric and that this calculation is ok.  Therefore it is provided as a subroutine.

\subsection{Element-by-element product}
\index{Vector!arithmetic!element by element product}
\index{Matrix!arithmetic!element by element product}
\index{DiagMatrix!arithmetic!element by element product}
\index{UpperTriMatrix!arithmetic!element by element product}
\index{BandMatrix!arithmetic!element by element product}
\index{SymMatrix!arithmetic!element by element product}
\index{SymBandMatrix!arithmetic!element by element product}

The two usual kinds of multiplication for vectors are the inner product and 
the outer product, which result in a scalar and a matrix respectively.
However there is also a third kind of multiplication that is sometimes needed where
each element in a vector is multiplied by the
corresponding element in another vector: $v(i) = v(i) * w(i)$.

There are two functions that should provide all of this kind of functionality
for you:
\begin{tmvcode}
ElementProd(T x, const GenVector<T1>& v1, const VectorView<T>& v2);
AddElementProd(T x, const GenVector<T1>& v1, const GenVector<T2>& v2,
      const VectorView<T>& v3)
\end{tmvcode}
The first performs $v_2(i) = x * v_1(i) * v_2(i)$, and the second performs
$v_3(i) = v_3(i) + x * v_1(i) * v_2(i)$ for $i = 0 ... (N-1)$ (where $N$ is the 
size of the vectors).

There is no operator overloading for \tt{Vector}s that would be equivalent to 
these expressions.
But they are actually equivalent to the following:
\begin{tmvcode}
v2 *= x * DiagMatrixViewOf(v1);
v3 += x * DiagMatrixViewOf(v1) * v2;
\end{tmvcode}
respectively.  In fact, these statements inline to the above function calls
automatically.  Depending on you preference and the meanings of your vectors,
these statements may or may not be clearer as to what you are doing.

There are also corresponding functions for \tt{Matrix} and for each of the special
matrix types:
\begin{tmvcode}
ElementProd(T x, const GenMatrix<T1>& m1, const MatrixView<T>& m2);
AddElementProd(T x, const GenMatrix<T1>& m1, const GenMatrix<T2>& m2,
      const MatrixView<T>& m3);
\end{tmvcode}
Likewise for the other special matrix classes.  The first performs 
$m_2(i,j) = x * m_1(i,j) * m_2(i,j)$, and the second performs
$m_3(i,j) = m_3(i,j) + x * m_1(i,j) * m_2(i,j)$ for every $i,j$ in the matrix.

These don't have any \tt{DiagMatrixViewOf} version, since the corresponding 
concept would require a four-dimensional tensor, and the TMV library
just deals with one- and two-dimensional objects.

The matrices all have to be the same size and shape, but can have any 
(i.e. not necessarily the same) storage method.  However, the routines are faster
if the matrices use the same storage.

\subsection{BaseMatrix views}
\index{BaseMatrix!views of}
\index{BaseMatrix!copy of}
\index{BaseMatrix!inverse of}

If you are dealing with objects that are only known to be \tt{BaseMatrix}es
(i.e. they could be a \tt{Matrix} or a \tt{DiagMatrix} or a \tt{SymMatrix}, etc.),
then methods like \tt{m.Transpose()}, \tt{m.View()}, and such
can't know what kind of object to return.
So these methods can't be defined for a \tt{BaseMatrix}.  

Instead, we have the following virtual methods, 
which are available to a \tt{BaseMatrix}
object and are defined in each specific kind of matrix to return a pointer
to the right kind of object:
\begin{tmvcode}
std::auto_ptr<tmv::BaseMatrix<T> > m.NewCopy()
std::auto_ptr<tmv::BaseMatrix<T> > m.NewView()
std::auto_ptr<tmv::BaseMatrix<T> > m.NewTranspose()
std::auto_ptr<tmv::BaseMatrix<T> > m.NewConjugate()
std::auto_ptr<tmv::BaseMatrix<T> > m.NewAdjoint()
std::auto_ptr<tmv::BaseMatrix<T> > m.NewInverse()
\end{tmvcode}
\tt{NewCopy} and \tt{NewInverse} create new storage to store a copy of the 
matrix or its inverse, respectively.  The other four just return views of the current 
matrix.

\subsection{Iterators}
\index{Vector!iterators}

We mentioned that the iterators through a \tt{Vector} are:
\begin{tmvcode}
typename tmv::Vector<T>::iterator
typename tmv::Vector<T>::const_iterator
typename tmv::Vector<T>::reverse_iterator
typename tmv::Vector<T>::const_reverse_iterator
\end{tmvcode}
just like for standard library containers.  The specific types to which these
typedefs refer are:
\begin{tmvcode}
tmv::VIt<T,tmv::Unit,tmv::NonConj>
tmv::CVIt<T,tmv::Unit,tmv::NonConj>
tmv::VIt<T,tmv::Step,tmv::NonConj>
tmv::CVIt<T,tmv::Step,tmv::NonConj>
\end{tmvcode}
respectively.

\tt{VIt} is a mutable-iterator, and \tt{CVIt} is a const-iterator.  \tt{Unit} 
indicates that the step size is 1, while \tt{Step} allows for any step size
between successive elements (and is therefore slower).  For the reverse
iterators, the step size is -1.

This can be worth knowing if you are going to be optimizing code that uses
iterators of \tt{VectorView}s.
This is because their iterators are instead:
\begin{tmvcode}
tmv::VIter<T>
tmv::CVIter<T>
\end{tmvcode}
which always check the step size (rather than assuming unit steps) and always
keep track of a possible conjugation.

If you know that you are dealing with a view that is not conjugated, you can 
convert your iterator into one of the above \tt{VIt} or \tt{CVIt} types, which will be 
faster, since they won't check the conjugation bit each time. 

Likewise, if you
know that it {\em is} conjugated, then you can use \tt{tmv::Conj} for the 
third template parameter above.  This indicates that the vector view really
refers to the conjugates of the values stored in the actual memory locations.

Also, if you know that your view has unit steps between elements, converting to 
an iterator with \tt{tmv::Unit} will iterate faster.  It is often faster to check
the step size once at the beginning of the routine and convert to a unit-step
iterator if possible.

All of these conversions can be done with a simple cast or constructor, such as:
\begin{tmvcode}
if (v.step() == 1) {
    for(VIt<float,Unit,NonConj> it = v.begin(); it != v.end(); it++)
        (*it) = sqrt(*it);
} else {
    for(VIt<float,Step,NonConj> it = v.begin(); it != v.end(); it++)
        (*it) = sqrt(*it);
}
\end{tmvcode}

Regular \tt{Vector}s are always \tt{Unit} and \tt{NonConj}, so those iterators
are already fast without using the specific \tt{VIt} names. 
That is, you can just use \tt{Vector<T>::iterator} rather than \tt{VIt<T,Unit,NonConj>}
without any performance drop.

\subsection{Direct memory access}
\index{Vector!direct access to memory}
\index{SmallVector!direct access to memory}
\index{Matrix!direct access to memory}
\index{DiagMatrix!direct access to memory}
\index{UpperTriMatrix!direct access to memory}
\index{BandMatrix!direct access to memory}
\index{SymMatrix!direct access to memory}
\index{SymBandMatrix!direct access to memory}

We provide methods for accessing the memory of a matrix or vector directly.
This is especially useful for meshing the TMV objects with other libraries
(such as BLAS or LAPACK).  But it can also be useful for writing some
optimized code for a particular function.  

The pointer to the start of the memory for a vector can be obtained by:
\begin{tmvcode}
T* v.ptr()
const T* v.cptr() const
\end{tmvcode}
\index{Vector!methods!ptr}
\index{Vector!methods!cptr}

Using the direct memory access
requires that you know the spacing of the elements in memory and
(for views) whether the view is conjugated or not.  So we also provide:
\begin{tmvcode}
int v.step() const
bool v.isconj() const
\end{tmvcode}
\index{Vector!methods!step}
\index{Vector!methods!isconj}

For matrices, the corresponding routines actually return the upper-left element
of the matrix.  For some matrices, (e.g. \tt{BandMatrix<T,DiagMatrix>}) 
this is not actually the first element in memory.  We also need to know the 
step size in both directions:
\begin{tmvcode}
T* m.ptr()
const T* m.cptr() const
int m.stepi() const
int m.stepj() const
bool m.isconj() const
bool m.isrm() const
bool m.iscm() const
\end{tmvcode}
\index{Matrix!methods!ptr}
\index{Matrix!methods!cptr}
\index{Matrix!methods!stepi}
\index{Matrix!methods!stepj}
\index{Matrix!methods!isconj}
\index{Matrix!methods!isrm}
\index{Matrix!methods!iscm}
\index{DiagMatrix!methods!ptr}
\index{DiagMatrix!methods!cptr}
\index{DiagMatrix!methods!isconj}
\index{UpperTriMatrix!methods!ptr}
\index{UpperTriMatrix!methods!cptr}
\index{UpperTriMatrix!methods!stepi}
\index{UpperTriMatrix!methods!stepj}
\index{UpperTriMatrix!methods!isconj}
\index{UpperTriMatrix!methods!isrm}
\index{UpperTriMatrix!methods!iscm}
\index{BandMatrix!methods!ptr}
\index{BandMatrix!methods!cptr}
\index{BandMatrix!methods!stepi}
\index{BandMatrix!methods!stepj}
\index{BandMatrix!methods!isconj}
\index{BandMatrix!methods!isrm}
\index{BandMatrix!methods!iscm}
\index{SymMatrix!methods!ptr}
\index{SymMatrix!methods!cptr}
\index{SymMatrix!methods!stepi}
\index{SymMatrix!methods!stepj}
\index{SymMatrix!methods!isconj}
\index{SymMatrix!methods!isrm}
\index{SymMatrix!methods!iscm}
\index{SymBandMatrix!methods!ptr}
\index{SymBandMatrix!methods!cptr}
\index{SymBandMatrix!methods!stepi}
\index{SymBandMatrix!methods!stepj}
\index{SymBandMatrix!methods!isconj}
\index{SymBandMatrix!methods!isrm}
\index{SymBandMatrix!methods!iscm}
The step in the ``down'' direction along a column is \tt{stepi}, and the step to 
the ``right'' along a row is \tt{stepj}.
The last two check if a matrix is \tt{RowMajor} or \tt{ColMajor} respectively.

For band matrices, there are also:
\begin{tmvcode}
int m.diagstep() const
bool m.isdm() const
\end{tmvcode}
\index{BandMatrix!methods!diagstep}
\index{BandMatrix!methods!isdm}
\index{SymBandMatrix!methods!diagstep}
\index{SymBandMatrix!methods!isdm}
which return the step along the diagonal and whether the matrix is \tt{DiagMajor}.

For symmetric/hermitian matrices, there are some more methods:
\begin{tmvcode}
bool m.isherm()
bool m.issym()
bool m.isupper()
\end{tmvcode}
\index{SymMatrix!methods!isherm}
\index{SymMatrix!methods!issym}
\index{SymMatrix!methods!isupper}
\index{SymBandMatrix!methods!isherm}
\index{SymBandMatrix!methods!issym}
\index{SymBandMatrix!methods!isupper}
The first two both return \tt{true} for real symmetric matrices, but 
differentiate between hermitian and symmetric varieties for complex types.
The last one tells you whether the actual elements to be accessed are stored
in the upper triangle half of the matrix (true) or the lower (false).

\subsection{``Linear'' views}
\label{linearviews}

Our matrices generally store the data contiguously in memory with all of the 
methods like \tt{row} and \tt{col} returning the appropriate slice through the
data.  Occasionally, though, it can be useful to treat the whole matrix
as a single vector of elements.  We use this internally for implementing routines
like \tt{SetAllTo} and matrix addition, among others.  These are faster than
accessing the data in ways that use the actual matrix structure.

Anyway, this same kind of access may be useful for some users of the library, 
so the following two methods are available:
\begin{tmvcode}
tmv::VectorView<T> m.LinearView()
tmv::ConstVectorView<T> m.ConstLinearView()
bool m.CanLinearize()
\end{tmvcode}
\index{Matrix!view as a contiguous Vector}
\index{Matrix!methods!LinearView}
\index{Matrix!methods!ConstLinearView}
\index{Matrix!methods!CanLinearize}
These return a view to the elements of a \tt{Matrix} as a single vector.  
It is always allowed for an actual \tt{Matrix}.  For a \tt{MatrixView} 
(or \tt{ConstMatrixView}), it is only allowed if all of the elements in the 
view are in one contiguous block of memory.  The helper function 
\tt{m.CanLinearize()} returns whether or not the first two methods are legal.

The same methods are also defined for \tt{BandMatrix} (and corresponding views).
In this case, there are a few elements in memory that are not necessarily
defined, since they lie outside of the actual band structure, so some care
should be used depending on the application of the returned vector views.  
(For example, one cannot compute things like the
minimum or maximum element this way, since the undefined elements may
have very large or small values which would corrupt this calculation.)

The triangular and symmetric matrices have too much memory that is not
actually used by the matrix for these to be very useful, so we do not provide them.
When we eventually implement the packed storage varieties, these methods will
be provided for those.

Along the same lines is another method for a \tt{Vector}:
\begin{tmvcode}
tmv::VectorView<RT> v.Flatten()
\end{tmvcode}
\index{Vector!methods!Flatten}
This returns a real view to the real and imaginary elements of a complex \tt{Vector}. 
The initial \tt{Vector} is required to have unit step.  The returned view has twice the 
length of \tt{v} and also has unit step.

This probably isn't very useful for most users either, but it is useful internally,
since it allows code such as:
\begin{tmvcode}
tmv::Vector<complex<double> > v(500);
[...]
v *= 2.3;
\end{tmvcode}
to call the BLAS routine \tt{dscal} with \tt{x=2.3}, rather than \tt{zscal}
with \tt{x=complex<double>(2.3,0.0)}, which would be slower.

\newpage
\section{Obtaining and Compiling the Library}
\index{Installation}

This code is licensed using the Gnu General Public License.  See 
\S\ref{license} below for more details.

\subsection[Basic installation]
{Basic installation \textnormal{(But see also \S\ref{scons} and \S\ref{cmake} below)}}
\label{install}
The following are step-by-step instructions on how to obtain the code for the library,
set it up for your system, and compile it:
\begin{enumerate}

\item
\index{Installation!downloading code}
Go to \website\ for a link to a tarball with all of the source code, and copy
it to the directory where you want to put the TMV library.

\item
Unpack the tarball:\\
\texttt{gunzip tmv\tmvversion .tar.gz}\\
\texttt{tar xf tmv\tmvversion .tar}

This will make a directory called \texttt{tmv\tmvversion} with the subdirectories:
\texttt{doc}, \texttt{examples}, \texttt{include}, \texttt{lib}, \texttt{src} and \texttt{test} 
along with the files \tt{README}, \tt{INSTALL} and others
in the top directory.

\item
Edit the makefile.
\index{LAPACK!setup makefile}
\index{BLAS!setup makefile}
\index{Installation!setup makefile}

The start of the makefile lists 5 things to specify: the compiler,
the include directories, the other flags to send to the compiler, 
any necessary BLAS/LAPACK linkage flags, and the installation directory.
The default setup is:
\begin{verbatim}
CC= g++
INCLUDE= -Iinclude
CFLAGS= $(INCLUDE) -O -DNOBLAS -DNDEBUG
BLASLIBS=
PREFIX=/usr/local
\end{verbatim}
but you will probably want to change this\footnote{
The original version of the \texttt{Makefile}
is copied in the file \texttt{Makefile.orig} in case your copy gets messed up, and you want to 
go back to the original.}.

This default setup will compile using g++ without any BLAS or LAPACK library and 
with debugging turned off. 
This setup should work on any system with gcc, although it almost certainly 
won't be as fast as using an optimized BLAS library and/or a LAPACK library.

You should edit these so that:
\begin{itemize}
\item \texttt{CC} specifies the C++ compiler to use.
\item \texttt{INCLUDE} specifies the directories for any BLAS and LAPACK header
files you want to include. (You should leave \tt{-Iinclude} there as well.)
\item \texttt{CFLAGS} contains any compiler flags you want.
(See below for TMV-specific flags to consider.)
\item \texttt{BLASLIBS} specifies the libraries required for linking your BLAS and LAPACK libraries.
\item \texttt{PREFIX} specifies the directory for \texttt{make install} to copy the files to.
\end{itemize}

After these lines, there are several commented-out examples for different systems 
using various BLAS
and LAPACK versions, showcasing some of the compiler options described below,
and giving examples of what you need for several common (or at least representative) systems.
If you have a system similar to one of these, then it should be a good 
starting point for you to figure out what you want to use.

See the next section below for a complete list of compiler flags that control how the TMV library is built.

\item{(advanced usage) Edit Inst and Blas files}

By default, the library will include instantiations of all classes and 
functions
that use either \texttt{double} or \texttt{float} (including complex versions of each).  
There are a few flags, such as 
\texttt{-DNO\_INST\_FLOAT} and \texttt{-DINST\_LONGDOUBLE},
that change this as described below.  
But if you want to compile routines for some other class, such as a user-defined
\texttt{MyQuadPrecisionType} class, then you will need to modify the file \texttt{TMV\_Inst.h} 
(in the \texttt{src} directory).
You simply need to add the lines:
\begin{tmvcode}
#define T MyQuadPrecisionType
#include InstFile
#undef T
\end{tmvcode}
\index{User-defined types}
to the end of the file before compiling.  (Obviously, you should
replace \tt{MyQuadPrecisionType} with whatever
type you want to be instantiated in the library.)

Also, the file \texttt{TMV\_Blas.h} (also in the \texttt{src} directory)
sets up all the BLAS and LAPACK calling structures,
as well as the necessary \texttt{\#include} statements.  
So if the BLAS or LAPACK options aren't working 
for your system, you may need to edit these files as well.
This is especially true if your BLAS or LAPACK versions are not one of 
ATLAS, CLAPACK, MKL, or ACML.  The comments at the beginning of 
\texttt{TMV\_Blas.h} gives instructions on how to set up the file for 
other installations.
\index{LAPACK!setup TMV\_Blas.h file}
\index{BLAS!setup TMV\_Blas.h file}

\item
Type: \\
\texttt{make libs}

This will make the TMV libraries, \texttt{libtmv.a} and \texttt{libtmv\_symband.a},
which will be located in the directory \texttt{lib}.

\item
(optional) Next type:\\
\texttt{make test}

This will make three executables called \texttt{tmvtest1}, 
\texttt{tmvtest2} and \texttt{tmvtest3} in the \texttt{bin} directory.

Warning: this step may take a long time to finish, since
the test code tests
for lots of different combinations of matrix types in all the various 
arithmetic operators.
So plan on letting this run
for a couple of hours (depending on your machine and compiler of course).

Then you should run the three test suites.
They should output a bunch of lines reading \texttt{[{\em Something}] passed all tests}.
If one of them ends in a line that starts with \texttt{Error}, 
then please send an email to \myemail\ about the problem including what compiler
you are using, some details about your system, and what (if any) BLAS and LAPACK
libraries you are linking to.

You may also want to make the example programs by typing:\\
\texttt{make examples}

This will make five executables called \texttt{vector}, \texttt{matrix}, \texttt{division},
\texttt{bandmatrix}, and \texttt{symmatrix}.
These programs, along with their corresponding source code in the \texttt{examples}
directory, give concrete examples of some of the common things you might want
to do with the TMV library.  They don't really try to be comprehensive,
but they do give a pretty good overview of the main features,
so looking at them may be a useful way to get started programming with TMV.

\item
(optional) Next type:\\
\texttt{make install}

This will copy the header files and the libraries to \texttt{PREFIX/include} and 
\texttt{PREFIX/lib}.  The default \texttt{PREFIX} is \texttt{/usr/local}, but you can change
this in the \texttt{Makefile} to some other installation directory if you want.

\item
Compile you program

Each \texttt{.cpp} file that uses TMV will need to have \\
\tt{\#include "TMV.h"}\\
at the top.
\index{TMV.h}
If you are using any of the special matrix types (other than diagonal and triangular),
then you also need to include their particular header files as well. (e.g.
\tt{TMV\_Band.h}, \tt{TMV\_Small.h}, etc.)
\index{TMV\_Band.h}
\index{TMV\_Small.h}
\index{TMV\_Sym.h}
\index{TMV\_SymBand.h}

If you did not install the header files (with \texttt{make install} above) to 
a directory that is in the standard path for header files, then 
you will need to use the compile flag
\texttt{-I[tmvdir]/include} when making the object file to tell the 
compiler where the TMV header files are.

For the linking step, you need to compile with the flag
\texttt{-ltmv}.
If you are using band, symmetric or symmetric band matrices
in your code, then you will 
need to link with the flags \texttt{-ltmv\_symband -ltmv} (in that order).
And if you did not install the libraries to a directory in your linking path,
then you need to include the flag
\texttt{-L[tmvdir]/lib} to tell the linker where the libraries are.

If you are using BLAS and/or LAPACK calls from the TMV code,
then you will also need to 
link with their libraries.  
For example, for my version of Intel's Math Kernel LIbrary, I use 
\texttt{-lmkl\_lapack -lmkl\_ia32 -lguide -lpthread}.  
For ATLAS,
I use \texttt{-llapack -lcblas -latlas}.  For your specific installation, you may need 
the same thing, or something slightly different, including possibly
\texttt{-L} flags to indicate where the BLAS or LAPACK libraries are located.
\index{LAPACK!linking}
\index{BLAS!linking}

\end{enumerate}

\subsection{Compiler flags}
\index{Installation!compiler flags}

Here are some compiler define flags to consider using:
\begin{itemize}
\item
\texttt{-DNDEBUG} will turn off debugging.  My recommendation is to leave debugging
on, since it doesn't slow things down too much.  Then when you decide to 
export a version of the code that needs to be as fast as possible, recompile
with this flag.  That said, the internal TMV code should be pretty well debugged
when you get it, so for compiling the library, you could go ahead and use this flag.
I keep two versions of the library for my own use - one with and one without 
this flag.  Then when I encounter a problem I can try linking to the debugging 
version to make sure it's not a bug in the TMV code.
\item
\texttt{-DNO\_INST\_FLOAT} will not instantiate any \tt{<float>} classes or routines.
\item
\texttt{-DNO\_INST\_DOUBLE} will not instantiate any \tt{<double>} classes or routines.
\item
\texttt{-DNO\_INST\_COMPLEX} will not instantiate any complex classes or routines.
\item
\texttt{-DINST\_LONGDOUBLE} will instantiate \tt{<long double>} classes and routines.
\item
\texttt{-DINST\_INT} will instantiate \tt{<int>} classes and routines.
\end{itemize}
The next flags specify what BLAS and/or LAPACK libraries to use (if any):
\index{LAPACK!compiler flags}
\index{BLAS!compiler flags}
\begin{itemize}
\item
\texttt{-DNOBLAS} will not call any external BLAS or LAPACK routines
\item
\texttt{-DNOLAP} will not call any external LAPACK routines
\item
\texttt{-DATLAS} will set up the BLAS calls as defined by ATLAS.  And (if -DNOLAP
is not specified), it will also call the several LAPACK routines
provided by ATLAS.
\item
\texttt{-DCBLAS} will set up the BLAS calls using the notation provided by the 
CBLAS interface to BLAS.  Many systems have a library like this installed as 
\texttt{/usr/local/lib/libcblas.a}, or some similar location,
so it is worth looking to see if you have this before
trying to install something new.
\item
\texttt{-DFBLAS} will set up the BLAS calls using a library that was designed to be called
from Fortran programs rather than C.  
Most BLAS installations also install the C interface, but if your
BLAS does not have this (e.g. GotoBLAS), then this flag should work.  
Since there is often no
header file for a Fortran BLAS library, we include the file \texttt{fblas.h} in the TMV 
\texttt{include} directory that should work.
\item
\texttt{-DCLAPACK} will set up the LAPACK calls for the CLAPACK distribution.  I 
find this version easier to get installed than the Fortran LAPACK distribution,
so I would recommend using this if you don't already have a version of LAPCK
installed somewhere on you system.  
Defining both ATLAS and CLAPACK will use
the CLAPACK version for all LAPACK routines, including the ones also provided by 
ATLAS.  That is, ATLAS will only be used for its BLAS routines.  If you want
the ATLAS versions of its few LAPACK routines instead, the ATLAS 
installation instructions describe a way to get them into the CLAPACK library.
Also, you should make sure the \texttt{-I} directories lists the directory with the CLAPACK
version of clapack.h before the ATLAS version.
\item
\texttt{-DFLAPACK} will set up the LAPACK calls for a distribution that
was designed to be called from Fortran programs rather than C.  Since such distributions
do not provide a header file, we provide \texttt{flapack.h} in the TMV \texttt{include} directory.
\item
\texttt{-DMKL} will call all the external BLAS and LAPACK routines as defined by the
Intel Math Kernel Library.  You should specify the directory with \texttt{mkl.h} with a 
\texttt{-I} flag if it is not installed in your include path.
\item
\texttt{-DACML} will call all the external BLAS and LAPACK routines as defined by the 
AMD Core Math Library.  You should specify the directory with \texttt{acml.h} with a 
\texttt{-I} flag if it is not installed in your include path.
\end{itemize}
The next set of compiler defines are not usually necessary.  But
if you have problems, these might be useful:
\begin{itemize}
\item
\texttt{-DXTEST} will do extra testing in the test suite, as well as add a few $O(N)$
time assert statements.  (Most of the assert statements that are normally run only take
$O(1)$ time.)  I always do tests with this turned on before releasing a version, but the
executable gets quite large, as do many of the \texttt{TMV\_Test*.o} files.  Plus it takes
a lot longer to compile.  So the default is to not use it.

\texttt{XTEST} mostly does additional checks
of the algorithms for different specific matrix pairs in the various
arithmetic operations to test possible failure modes.  If the shorter test suite
(i.e. without \texttt{XTEST}) works ok for you, you should be fine.

\item
\texttt{-DMEMTEST} will do extra testing of the memory usage to look for memory leaks
and other memory errors.  Again, there is probably no need for you to use this
flag.  But if you think there is a memory leak in TMV, this could help you find it, 
in which case please let me know.

\item
\texttt{-DNOSTEGR} specifies that the LAPACK algorithm called
\tt{dstegr} (or \tt{sstegr} for \tt{<float>}) should not be used for symmetric
eigenvector calculation.  In this case, the divide-and-conquer algorithm,
named \tt{dstedc} (or \tt{sstedc}) will be used instead.
\index{Singular Value Decomposition!LAPACK ?stegr}
\index{Eigenvalues!LAPACK ?stegr}
\index{Warnings!LAPACK ?stegr}
\index{LAPACK!Problems with ?stegr}

I used to have problems with the \tt{stegr} algorithm producing incorrent
results, but I figured out how to detect the problem and now
the TMV code starts by calling \tt{stegr}, and then if it fails it calls
the slower, but more robust, \tt{stedc} routine.
If you want to avoid this behavior and always use the \tt{stedc} algorithm,
you can compile with this flag.
\item
\texttt{-DNOGEQP3} specifies that the LAPACK algorithm called
\tt{dgeqp3} (or its variants) should not be used for the strict QRP decomposition.
In this case, the native TMV code will be used instead.
\index{QRP Decomposition!LAPACK ?geqp3}
\index{LAPACK!Problems with QRP decomposition}

I have found that the LAPACK code for \tt{?geqp3} does not always
produce an $R$ matrix
with strictly decreasing elements along the diagonal.  So if this is important
for you, then you should use this flag.
\item
\texttt{-DNOWORKQUERY} specifies that your LAPACK library does not support
work size queries.  There are a few LAPACK routines that require workspace whose
optimal size depends on details of the L1 cache size of your machine.  Normally,
you can pass the routine a work size of -1 to indicate that you are doing a workspace
query, in which case the routine returns the optimal value to use.  This is the normal
mode of operation for TMV.  

However, some older LAPACK distributions do not support
this process.  The example I found was the MKL with icc version 9.0.
So if you get errors similar to \\
\texttt{TMV Error: info < 0 returned by LAPACK function dormqr}\\
then this flag will compile without the workspace queries, instead just
using a good guess for the optimal size.

\item
\texttt{-DNOMIX\_SMALL} will avoid all arithmetic that mixes \tt{SmallMatrix} and \tt{SmallVector} 
with regular \tt{Matrix} or \tt{Vector} in the test program \texttt{tmvtest3}.
Some BLAS libraries -- specifically GotoBLAS, possibly others -- 
don't work with this kind of mixing, so this flag allows you to test
the rest of the \tt{Small} arithmetic if the mixing is causing problems for your
system.  

\item
\texttt{-DNOSTL} uses some workarounds for segments of code that use the STL
library, but which didn't work for one of my compilers.  I'm pretty sure it is
because the compiler wasn't installed correctly, so I don't think you
should really ever need to use this flag.  
But in any case, it 
will use a median-of-three quicksort algorithm for sorting
rather than the standard library's sort.  And it manually reads strings 
using character reads, rather than using the \tt{>>} operator.  

\item
\texttt{-DXDEBUG} will do different extra (much slower) debugging.  This one checks for
incorrect results from the various algorithms by doing things the simple slow way
and comparing the results to the fast blocked or recursive or in-place version
to make sure the answer isn't (significantly) different.  
I use this one a lot when debugging
new algorithms, usually on a file-by-file basis.  Again, you shouldn't need this for
an official release version.  But if you do get wrong answers for something, you could
use this to try to find the problem.

\item
\texttt{-DTMV\_BLOCKSIZE=NN} will change the block size used by some routines.  
The current value is 64, which is good for many computers.  
The optimal value will depend on the size of your
CPU's L1 cache.  So if you want to try to tune the algorithms, you can 
modify this value to something more appropriate for your computer.
However, you are probably better off just using an optimized BLAS and LAPACK
library and let TMV call that instead.
\end{itemize}

\subsection{Alternate SCons installation}
\label{scons}
\index{Installation!scons}

With this release of TMV, I am introducing a new installation method that should be 
easier in most cases.  The intent is for it to look for what BLAS and LAPACK libraries
you have on your system and automatically compile with the appropriate flags and linkage.

So far, this is still somewhat in the development stage, so it might not find your LAPACK
library correctly, in which case, you will need to stick with the above \texttt{Makefile} installation.
But I have tested it on quite a few BLAS and LAPACK varieties, so
if you have SCons installed on your system, 
and you want as simple an installation procedure as possible, you might want to 
try this first.

If you do try this installation method, I would appreciate it if you could inform me of any problems 
you have.

\begin{enumerate}
\item Download and unpack the tarball as described above.

\item Make sure you have SCons installed on your system.  
It is available for free from \url{http://www.scons.org/}.  (It is a
very quick installation if you have Python installed.)

\item Type \\
\texttt{scons}

This will make the libraries \texttt{libtmv.a} and \texttt{libtmv\_symband.a}
and put them into the directory \texttt{lib}.  Like with \texttt{make}, you can add
the flag \texttt{-j4} to use 4 (or whatever number of) compilers simultaneously.
Also, the command \texttt{scons -h} will print some help information.

There are a number of command-line options that you might need (but try it with no flags
first -- it can often find everything automatically).  
The options are listed 
with their default value.  You change them simply by specifying a different value
on the command line.  For example:\\
\texttt{scons CXX=icpc WITH\_LONGDOUBLE=true}\\
If you need to run SCons multiple times (for example to compile the test suite or install
the libraries as described below), you only need to specify the new parameter values the first
time you run the SCons.
The program automatically saves your options and continues to use them until you
change a value again.
\begin{itemize}
\item \texttt{CXX=g++} specifies which C++ compiler to use.
\item \texttt{FLAGS=''} specifies the basic flags to pass to the compiler.  The default behavior is to
automatically choose good flags to use according to which kind of compiler you are using.
It has defaults for \texttt{g++}, \texttt{icpc} and \texttt{pgCC}.  If you are using a different compiler or don't
like the default, then you can specify this by hand.  
Remember to put the flags in quotes,
so the whitespace doesn't confuse the parser.  e.g. \texttt{scons FLAGS='-O3 -g'}
\item \texttt{DEBUG=false} specifies whether to keep the debugging assert statements in the compiled library code.  
\item \texttt{PREFIX=/usr/local} specifies where to install the library when running \texttt{scons install} (see below).
\item \texttt{WITH\_FLOAT=true} specifies whether to instantiate the \tt{<float>} templates.
\item \texttt{WITH\_DOUBLE=true} specifies whether to instantiate the \tt{<double>} templates.
\item \texttt{WITH\_LONGDOUBLE=false} specifies whether to instantiate the \tt{<long double>} templates.
\item \texttt{WITH\_INT=false} specifies whether to instantiate the \tt{<int>} templates.
\end{itemize}
The next flags set up the paths that SCons will use to try to find your BLAS and LAPACK libraries.
\begin{itemize}
\item \texttt{EXTRA\_PATH=''} specifies directories in which to search for executables (notably the compiler, although you can also just give the full path in the \texttt{CXX} parameter)
in addition to the standard locations such as \texttt{/usr/bin} and \texttt{/usr/local/bin}.
If you are giving multiple directories, they should be separated by colons.
\item \texttt{EXTRA\_INCLUDE\_PATH=''} specifies directories in which to search for header files
(such as the BLAS or LAPACK header files)
in addition to the standard locations such as \texttt{/usr/include} and \texttt{/usr/local/include}.
These directories are specified as \texttt{-I} flags to the compiler.
If you are giving multiple directories, they should be separated by colons.
\item \texttt{EXTRA\_LIB\_PATH=''} specifies directories in which to search for libraries
(such as the BLAS or LAPACK libraries)
in addition to the standard locations such as \texttt{/usr/lib} and \texttt{/usr/local/lib}.  
These directories are specified as \texttt{-L} flags to the linker.
If you are giving multiple directories, they should be separated by colons.
\item \texttt{IMPORT\_PATHS=false} specifies whether to import extra path directories from the environment variables: 
\texttt{PATH}, \texttt{C\_INCLUDE\_PATH}, \texttt{LD\_LIBRARY\_PATH} and \texttt{LIBRARY\_PATH}.
\item \texttt{IMPORT\_ENV=false} specifies whether to import the entire environment from the calling shell.
The default
is to start with a clean environment to be less susceptible to a particular user having an unusual set up.  But sometimes sysadmins set things up in non-standard ways and use
the environment variables to make everything work.  If this is the case, then 
\texttt{IMPORT\_ENV} should do the trick.  It imports the environment, but doe not add any
\texttt{-I} or \texttt{-L} flags when compiling.
\end{itemize}
The next options can be used to specify what BLAS and/or LAPACK libraries to use (if any),
overriding the default of using whatever libraries SCons chooses from searching through your path
and trying to link the libraries that it finds.  The \texttt{FORCE} options can be useful if SCons finds
a library before trying the one that you want, or if SCons fails in the linking step even though
the library should link successfully (I'm still not sure why this happens sometimes), or if
you want to compile for a library that requires different linking instructions than the 
ones that SCons tries\footnote{
If you have a case of needing different linking instructions, and your BLAS or LAPACK is a standard
installation on your machine (not some goofy personal installation that no one else will duplicate),
then let me know and I'll add it to the SConstruct file for the next release.}.
The \texttt{FORCE} options will try to test linking with the library requested,
but if it fails, then it will just give a warning message.
\index{LAPACK!scons options}
\index{BLAS!scons options}
\begin{itemize}
\item \texttt{WITH\_BLAS=true} specifies whether to look for and try to use a BLAS library.
\item \texttt{WITH\_LAPACK=true} specifies whether to look for and try to use a LAPACK library.
\item \texttt{FORCE\_MKL=false} forces the use of the Intel Math Kernel library. 
It requires the header file \texttt{"mkl.h"} to be found in your path. 
\item \texttt{FORCE\_ACML=false} forces the use of the AMD Core Math library.  
It requires the header file \texttt{"acml.h"} to be found in your path.
\item \texttt{FORCE\_GOTO=false} forces the use of the GotoBlas library.  
\item \texttt{FORCE\_ATLAS=false} forces the use of the ATLAS library (for BLAS).  
It requires the header file \texttt{"cblas.h"} to be found in your path.
\item \texttt{FORCE\_CBLAS=false} forces the use of a CBLAS library.
\item \texttt{FORCE\_FBLAS=false} forces the use of a Fortran BLAS library.
\item \texttt{FORCE\_CLAPACK=false} forces the use of the CLAPACk library. 
It requires the CLAPACK version of the header file \texttt{"clapack.h"} to be found in your path.
\item \texttt{FORCE\_ATLAS\_LAPACK=false} forces the use of the LAPACK portion
of the ATLAS Library.  
It requires the ATLAS version of the header file \texttt{"clapack.h"} to be found in your path.
\item \texttt{FORCE\_FLAPACK=false} forces the use of a Fortran LAPACK library.
\item \texttt{LIBS=''} specifies library flags to use for linking.  Because of the way SCons works,
these should omit the \texttt{-l} part of the flag, since SCons will add this to what is provided.
For example, to specify an alternate name for the CLAPACK library, use 
\texttt{scons LIBS=lapack\_LINUX}.  Multiple libraries here should be separated by 
whitespace and enclosed in quotes.
\end{itemize}
Finally, some miscellaneous options that you are less likely to need:
\begin{itemize}
\index{Installation!OpenMP}
\index{OpenMP}
\item \texttt{WITH\_OPENMP=false} specifies whether to use OpenMP to parallelize some parts of the code.
\item \texttt{STATIC=false} specifies whether to use static linkage.  Some systems have trouble with
dynamic linkage of libraries.  This usually indicates that something is installed incorrectly, but
it can be easier to just use static linkage when you compile as a workaround.  This flag does this
for the test suite executables.
\item \texttt{XTEST=false} specifies whether to include extra tests in the test suite.
\item \texttt{MEM\_TEST=false} specifies whether to include extra memory tests in the library and test suite.
\item \texttt{SMALL\_TESTS=false} specifies whether to make the smaller test suite programs: \texttt{tmvtest1a}, \texttt{tmvtest1b}, etc. 
\end{itemize}

When SCons starts up, it will look through the standard paths, along with any extra paths you have
specified with the above options, to find BLAS
and LAPACK libraries.  This can sometimes require a few iterations to get working correctly.  
You should look at the initial output from SCons to make sure it finds the correct BLAS
and LAPACK libraries that you think it should find.  Here is a sample output:\footnote{
This is the exact output that I get with the default options on my MacBook.}
\begin{verbatim}
$ scons
scons: Reading SConscript files ...

Using compiler: g++-4
compiler version: 4.3.2
Debugging turned off
Checking for MKL... no
Checking for ACML... no
Checking for GotoBLAS... no
Checking for CBLAS... yes
Using CBLAS
Checking for CLAPACK... no
Checking for Fortran LAPACK... yes
Using Fortran LAPACK
scons: done reading SConscript files.
scons: Building targets ...
\end{verbatim}
\vspace{-11pt}(\emph{Starts the actual compiling})

If a "\texttt{Checking for}..." line ends with \texttt{no}, even though you think that library is installed
on your computer, then it probably means that you need to tell SCons which directories
to search, in addition to the standard locations.  The most straightforward way to do this is
with the parameters \texttt{EXTRA\_INCLUDE\_PATH} and \texttt{EXTRA\_LIB\_PATH}.  
These are described in detail above.  See also \texttt{IMPORT\_ENV} and \texttt{IMPORT\_PATHS}.

\item (optional) Type\\
\texttt{scons test}
\index{Installation!test suite}

This will make the executables \texttt{tmvtest1}, \texttt{tmvtest2} and \texttt{tmvtest3} in
the \texttt{test} directory.  As described above, running each of these should output 
lines that look like \texttt{[Something] passed all tests}.  If the last line indicates
an error, you should email \myemail\ about the problem.

If you specify that \texttt{SMALL\_TESTS=true}, then the smaller test executables
\texttt{tmvtest1a}, \texttt{tmvtest1b}, \texttt{tmvtest1c}, \texttt{tmvtest2a}, \texttt{tmvtest2b}, \texttt{tmvtest2c},
\texttt{tmvtest3a}, \texttt{tmvtest3b}, \texttt{tmvtest3c}, and \texttt{tmvtest3d} will be made instead.
These perform the same tests as the larger test executables, but can be easier
for some linkers.

\item (optional) Type\\
\texttt{scons install}

This will install the necessary header files into the directory \texttt{/usr/local/include} and the libraries
into \texttt{/usr/local/lib}.  As mentioned above, you can also specify a different prefix 
with the command line option \texttt{PREFIX=}\emph{install-dir}.  A common choice for users
without \texttt{sudo} privileges is \texttt{PREFIX=$\sim$} which will install the library in \texttt{$\sim$/include}
and \texttt{$\sim$/lib}.

\end{enumerate}

\subsection{Alternate cmake installation}
\label{cmake}

CMake is another automated build tool.  I am not very well versed in it, but a user of the 
TMV library, Andy Molloy, sent me the \texttt{CMakeLists.txt} files that he used to compile TMV,
so I am distributing them for anyone else who wants
to use them.  It is an easy way to build a basic TMV library, but I am not sure how hard it is
to get it working with BLAS or LAPACK libraries\footnote{
I was about to start learning more about it when a colleague, Erin Sheldon, introduced me to
SCons, which I like a lot, so I haven't been motivated to go back to CMake and really flesh
it out like I did with the SCons installation script.}.
The basic usage is as follows:

\begin{enumerate}
\item Download and unpack the tarball as described above.

\item
Download CMake from \url{http://www.cmake.org/} if you do not already have it.

\item
In TMV directory, type\\
\texttt{cmake -i .}\\
or\\
\texttt{ccmake .}\\
for Unix, or run \texttt{CMakeSetup.exe} in Windows.

Answer the configuration questions and/or set the options listed.

\item
Type:\\
\texttt{make tmv}\\
\texttt{make tmv\_symband}\\
to make the libraries.

\item
(Optional) Type:\\
\texttt{make tmvtest1}\\
\texttt{make tmvtest2}\\
\texttt{make tmvtest3}\\
to make the test suites.

\end{enumerate}

I haven't incorporated most of the options that I added to the SCons installation above.
But I think all of that is possible with CMake, so if someone who is an expert in CMake wants
to do this and send me the CMake files for it, I'll include them in future releases.

\subsection{Installation using Microsoft Visual C++}

Using TMV with Microsoft Visual C++ is a bit different from the other compilers,
since it has a Windows framework for building programs, rather than a command 
line\footnote{
There is a way to do compile on the command line, but I suspect that will not be the 
usual way that people will want to use the TMV library.  If you are interested in compiling
this on the command line, I did have success installing SCons and then, within the
Visual Studio 2008 Command Prompt, using the command:
\texttt{C:$\backslash$Python26$\backslash$Scripts$\backslash$scons CXX=cl IMPORT\_ENV=true}.}.

\begin{enumerate}
\item
Download and unpack the tarball.  

There are many Windows utilities that can unpack the
tarball.  With IZArc (\url{http://www.izarc.org/}), for example, you right click on the file 
\texttt{tmv\tmvversion.tar.gz}
in Windows Explorer, select IZArc, then select Extract Here.  This should make a directory
called \texttt{tmv\tmvversion} which has all of the source code and other files for the TMV library.

\item
Start Microsoft Visual C++.  

I have Visual C++ 2008 Express Edition, so all instructions below
about menus and such refer that that edition.  I would assume that other editions have their 
menus arranged similarly, but there may be some differences.

\item
Open the ``Solution'' file \texttt{tmvtest1.sln}.

Each test suite needs to be made individually, so I'll give detailed instructions for the first
test program.  The other two are made the same way.

Go to the File menu.  Select Open.  Then select Project/Solution...
In the \texttt{tmv\tmvversion} directory, look for the file
\texttt{tmvtest1.sln} and select it.  

This includes a project for the first test program, \texttt{tmvtest1.vcproj}, and a project for
the main TMV library, \texttt{tmv.vcproj}.

\item
Select Debug or Release mode.

There are two modes for building programs in Visual C++: Debug and Release.  
You can choose either one.  Dubug mode will compile faster and 
Release mode will execute faster.  

When writing 
your own code, you will probably want to start with Debug mode and then switch to Release
when everything is working, so you will end up compiling the TMV library in both modes anyway.

You can select the mode in a pull down menu in the top button bar.  With my setup
(which I think is the default), it is directly to the right of a green arrow.

\item
Build \texttt{tmvtest1}.

Go to the Build menu.  Select Build Solution.

\item 
Run \texttt{tmvtest1}.

Go to the Debug menu.  Select Start Without Debugging.  (Of course, you can instead
choose Start Debugging if you'd rather, but that will make it run slower.)

A console window should open up to show the output of the program.  
When it is done, you should see the message,
"Press any key to continue . . . "

The previous output lines should all read \texttt{[{\em Something}] passed all tests}.
If the last line starts with \texttt{Error}, 
then please email \myemail\ about the problem.

\item
Repeat for \texttt{tmvtest2} and \texttt{tmvtest3}.  

Note: the solution file for \texttt{tmvtest2} includes the
project for the TMV library with symmetric and banded matrices, \texttt{tmv\_symband.vcproj}.

\item
Include \texttt{tmv.vcproj} (and \texttt{tmv\_symband.vcproj} if necessary) in the solution for your own project.

Any program that uses the TMV library needs to include the project \texttt{tmv.vcproj}.  To do this,
Go to the File, select Add, Existing Project...
In the \texttt{tmv\tmvversion} directory, look for the file
\texttt{tmv.vcproj} and select it.  If you are going to be using symmetric and/or banded matrices,
then you should also select \texttt{tmv\_symaband.vcproj}.

Then, in the Solution Explorer window, select your project.  Then go to the Project menu
and select Properties.  Select Common Properties, then Framework and References.  
Click the Add New Reference... button.  Select the TMV library (or libraries), and press OK.

Next you need to tell your program where to look for the TMV header files.  So on the Property Page,
select All Configurations at the top.  Then go to Configuration Properties, C/C++, General.
At the entry for Additional Include Directories, click the [...] button at the right.  Click on the folder icon
to add a new line, and then click the [...] button.  Browse to the 
\texttt{tmv\tmvversion} directory, and select the include directory.  Press OK three times.

Now you should be all set to use the TMV library in your program.

\end{enumerate}

\subsection{Known compiler issues}
\index{Installation!known problems}
\label{compilerissues}

%Done:
% pgCC ACML 
% pgCC NOBLAS
% pgCC NOBLAS OPENMP
% pgCC ATLAS FLAPACK
% pgCC ATLAS ATALS_LAP
% pgCC ATLAS NOLAP 
% pgCC ATLAS NOLAP OPENMP
% pgCC FBLAS FLAPACK
% pgCC ACML LONGDOUBLE,INT- X long double I/O errors.  Known error in pgCC 6.1.  Fixed. Kinda.

% g++3.4.6 ATLAS FLAPACK
% g++3.4.6 FBLAS FLAPACK
% g++3.4.6 NOBLAS

% icpc11.0 MKL
% icpc11.0 NOBLAS OPENMP
% icpc11.0 ATLAS ATLAS_LAP
% icpc11.0 NOBLAS LONGDOUBLE

% icpc10.1 MKL
% icpc10.1 NOBLAS OPENMP

% icpc9.1 FBLAS FLAPACK
% icpc9.1 MKL
% icpc9.1 NOBLAS OPENMP
% icpc9.1 MKL OPENMP

% icpc9.0 MKL X errors with lwork=-1 queries.  Fix: Use -DNOWORKQUERY
% icpc9.0 NOBLAS OPENMP

% g++4.0.2 NOBLAS

% g++4.1.2 MKL OPENMP X Nope.  Libraries don't mix correctly.
% g++4.1.2 MKL 
% g++4.1.2 NOBLAS OPENMP

% g++ 4.2.2 NOBLAS OPENMP

% g++4.3.2 NOBLAS OPENMP
% g++4.3.2 CBLAS FLAPACK
% g++4.3.2 CBLAS CLAPACK
% g++4.3.2 FBLAS FLAPACK
% g++4.3.2 FBLAS CLAPACK X couldn't link ?getri functions.  -- Documented as known issue.
% g++4.3.2 ATLAS ATLAS_LAP (errors in testmix, but test3 was ok.)
% g++4.3.2 GOTO NOLAP X errors mixing SmallMatrix with Matrix.  Avoid: Use -DNOMIX_SMALL
% g++4.3.2 ATLAS ATLAS_LAP OPENMP 

% g++4.3.1 CBLAS FLAPACK

% Apple g++4.0.1 CBLAS FLAPACK
% Apple g++4.2.1 NOBLAS OPENMP
% Apple g++4.2.1 ATLAS ATLAS_LAP OPENMP

% Visual C++ Windows UI
% Visual C++ cl with SCons


I have tested the code using the following compilers:\\
$\quad$\\
GNU's g++ -- versions 3.4.6, 4.0.2, 4.1.2, 4.2.2, 4.3.1 and 4.3.2\\
% chimaera g++, milhouse g++, lupus g++, Clarinet g++-4 (alt fink), Clarinet g++-4.3, Clarinet g++-4
Apple's g++ -- version 4.0.1, 4.2.1 \\
% Clarinet g++-4.0, g++-4.2
Intel's icpc -- versions 9.0, 9.1, 10.1 and 11.0\\
% milhouse, lupus, lupus (alt paths, static), des1
Portland's pgCC -- version 6.1\\
% chimaera pgCC
Microsoft's cl -- Visual C++ 2008 Express Edition\\
% Sousaphone
\index{Installation!g++}
\index{Installation!icpc}
\index{Installation!pgCC}
\index{Installation!Visual C++}

It should work with any ansi-compliant
compiler, but no guarantees if you use one other than these\footnote{
It does seem to be the case that 
every time I try the code on a new compiler, there is some issue that needs to be addressed.  
Either because the compiler fails to support some aspect of the C++ standard, or they enforce
an aspect that I have failed to strictly conform to.}.
  So if you do try to compile on a different compiler, 
I would appreciate it if you could let me know whether you were successful.  
Email me at \myemail.

There are a few issues that I have discovered when compiling with various 
versions of compilers, and I have usually come up with a work-around for
the problems.  So if you have a problem, check this list to see if a solution
is given for you.  

\begin{itemize}
\item {\bf Apple g++:}
\index{Installation!g++}
Older versions of Apple's version of g++ that they shipped with the Tiger OS did not work for 
compilation of the TMV library.  It was called version 4.0, but I do not remember the build number.
They seem to have fixed the problem with a later XCode update,
but if you have an older Mac and want to compile TMV on it and the native g++ 
is giving you trouble,
you should either upgrade to a newer Xcode distribution or download the real GNU gcc instead;  
I recommend using Fink (\url{http://fink.sourceforge.net/}).

\item {\bf g++ -O2, version 4,1 and 4.2:}
\index{Installation!g++}
It seems that there is some problem with the -O2 optimization of g++ versions 4.1.2 and 4.2.2
when used with TMV debugging turned on.  Everything compiles fine when I use
\texttt{g++ -O} or \texttt{g++ -O2 -DNDEBUG}.  But when I compile with \texttt{g++ -O2} (or \texttt{-O3}) without
\texttt{-DNDEBUG}, then the test suite fails, getting weird results for some arithmetic operations
that look like uninitialized memory was used.  

I distilled the code down to a small code snippet that still failed 
and sent it to Gnu as a bug report.
They confirmed the bug and suggested
using the flag \texttt{-fno-strict-aliasing}, which did fix the problems.

Another option, which might be a good idea anyway is to just use \texttt{-O} 
when you want a version that 
includes the TMV assert statements, and make sure to use \texttt{-DNDEBUG} 
when you want a more optimized version.

\item {\bf pgCC:}
\index{Installation!pgCC}
I only have access to pgCC version 6.1, so these notes only refer to that version.
\begin{itemize}
\item
Apparently pgCC does not by default support exceptions when compiled 
with openmp turned on.  
So if you want to use the parallel versions of the algorithms,
you need to compile with the flags \texttt{-mp --exception}.  This is a documented feature,
but it's not very obvious, so I figured I'd point it out here.

\item 
There was a bug in pgCC version 6.1 that was apparently fixed in version 7.0 where
\tt{long double} variables were not correctly written with \tt{std::ostream}.  The values
were written as either \tt{0} or \tt{-}.  So I have written a workaround in the code for
pgCC versions before version 7.0\footnote{
Thanks to Dan Bonachea for making available his file \texttt{portable\_platform.h},
which makes it particularly easy to test for particular compiler versions.},
where the \tt{long double} values are copied to 
\tt{double} before writing.  This works, but only for values that are so convertible.
If you have values that are outside the range representable by a \tt{double}, then 
you may experience overflow or underflow on output.

\end{itemize}

\item {\bf Borland's C++ Builder:}
\index{Installation!Borland C++ Builder}
I tried to compile the library with Borland's C++ Builder for Microsoft Windows
Version 10.0.2288.42451 Update 2, but it failed at fairly foundational aspects of the 
code, so I do not think it is possible to get the code to work.  However, if somebody wants
to try to get the code running with this compiler or some other Borland product, 
I welcome the effort and would
love to hear about a successful compilation (at \myemail).

\item {\bf Sun CC:}
\index{Installation!Sun CC}
I also tried to compile the library with Sun's CC compiler, version 5.3, 
and it didn't understand some of the template syntax used in TMV.  However,
this version is pretty old (2001), so it is likely that newer versions are more
compliant with the C++ standard (they are now up to version 5.9).  
Unfortunately, I do not have access to a newer version of CC yet, so I have not
been able try it.  As usual, if you have managed to compile TMV with
any version of Sun's CC, I would love to hear about it (at \myemail).

\item {\bf Memory requirements:}
The library is pretty big, so it can take quite a lot of memory to compile. 
For most compilers, it seems that a minimum of around 512K is required.
For compiling the test suite with the \texttt{-DXTEST} flag, more than 
2GB of memory is recommended.

\item {\bf Linker choking:}
Some linkers (e.g. the one on my old Mac G5) have trouble with the size 
of some of the test suite's executables, especially when compiled 
with \texttt{-DXTEST}.  If you encounter this problem, you can instead
compile the smaller test suites.  

The tests
in \texttt{tmvtest1} are split into \texttt{tmvtest1a}, \texttt{tmvtest1b} and \texttt{tmvtest1c}.
Likewise \texttt{tmvtest2} has \texttt{a}, \texttt{b} and \texttt{c} versions, and \texttt{tmvtest3}
has \texttt{a}, \texttt{b}, \texttt{c} and \texttt{d} versions.  These are 
compiled by typing \texttt{make test1a},~ \texttt{make test1b}, etc.  So if you want
to run the tests on a machine that can't link the full programs, these
smaller versions can help.   

With the SCons installation, you can 
type \texttt{scons test SMALL\_TESTS=true}.  Or you can make them one
at a time by typing \texttt{scons test1a},~ \texttt{scons test1b}, etc.

You might also try testing only one type at a time: First compile with \texttt{-DNO\_INST\_FLOAT},
and then with \texttt{-DNO\_INST\_DOUBLE} (or with SCons, use \texttt{WITH\_FLOAT=false}
and then \texttt{WITH\_FLOAT=true WITH\_DOUBLE=false}).  This cuts the size of the executables
in half, which can also help if the above trick doesn't work.  (I had to do this on one 
of my test systems for test2c.)

\index{Installation!test suite}

\item{\bf BLAS/LAPACK problems:}
There are a number of possible errors that are related to particular BLAS or LAPACK
distributions, or combinations thereof:
\begin{itemize}
\item{\bf Strict QRP decomposition fails:}
\index{LAPACK!Problems with QRP decomposition}
Some versions of the LAPACK function \tt{dgeqp3} do not produce the correct
result for $R$.  The diagonal elements of $R$ are supposed to be monotonically decreasing
along the diagonal, but sometimes this is not strictly true.  This probably varies among
implementations, so your version might always succeed.

But if this feature is important to you, then you can compile with the flag \texttt{-DNOGEQP3},
which will use the native TMV code for strict QRP decomposition for this 
rather than the LAPACK function (which is called \tt{?geqp3}).

\item{\bf Info $<$ 0 errors:}
\index{LAPACK!Problems with workspace queries}
If you get an error that looks something like:\\
\texttt{TMV Error: info < 0 returned by LAPACK function dormqr}\\
then this probably means that your LAPACK distribution does not support
workspace size queries.  The solution is to use the flag \texttt{-DNOWORKQUERY}.

\item {\bf Unable to link CLAPACK:}
\index{LAPACK!Problems linking CLAPACK}
I get a linking error on my Mac when combining CLAPACK with a Fortran-style BLAS library.
Specifically, it can't find the \texttt{?getri} functions.   Basically, the reason is that
this LAPACK library, libclapack.dylib, is designed to be used with the BLAS
library, libcblas.dylib, in the same directory, and that BLAS library has the \texttt{getri}
functions, even though they are properly LAPACK functions.
Anyway, it's basically a bug in the Apple XCode distribution of these files,
and the result is that if you use a different BLAS library, the linking fails.

The solution is either to use the Apple CBLAS library or to install your own CLAPACK
library.  If you do the latter, you will probably want to rename (or delete) these Mac
library files, since they are in the /usr/lib directory, and \tt{-L} flags usually can't take 
precedence over /usr/lib.

\item {\bf Segmentation faults with \tt{SmallMatrix}:}
\index{LAPACK!Problems with SmallMatrix}
\index{SmallMatrix!mixing with regular Matrix, Vector}
\index{SmallVector!mixing with regular Matrix, Vector}
My version of the GotoBLAS library seems to have trouble with arithmetic statements that
mix \tt{SmallMatrix} or \tt{SmallVector} with \tt{Matrix} or \tt{Vector}.  I think the
problem stems from the fact that the \tt{Small} objects allocate their memory on the
stack, while the regular objects allocate their memory on the heap.  
One of the ATLAS installations I tested also had similar trouble.

If your BLAS distribution has problems running \texttt{tmvtest3} (which is the one
that runs the tests of 
\tt{SmallMatrix} and \tt{SmallVector}), then you should try
compiling with the flag \texttt{-DNOMIX\_SMALL}
to tell the code to skip the tests that mix \tt{Small} and regular objects.
And of course, you will also want to remember to watch out for this in your own code 
if you use \tt{SmallMatrix} and \tt{SmallVector}.
The SCons installer automatically uses this flag when using GotoBLAS.

If you want to test the mixed arithmetic specifically, you can use \texttt{scons testmix}
to compile the test program \texttt{tmvtestmix}.  I am planning on having scons
compile and run this program before compiling \texttt{tmvtest3} to see whether it needs
the flag, but I haven't figured out how to do that yet.

\end{itemize}

\item {\bf Standard Template Library:}
In a few places, the TMV code uses standard template library algorithms.
Normally these work fine, but a couple of the compilers I tested the code on
didn't seem to have the STL correctly installed. 

On one computer, the linker complained 
that it couldn't find the code for sort, even though sort should be completely inlined,
so the code should already have been in \texttt{TMV\_Vector.o}, where I use it.
Similarly, it had trouble linking the string read commands needed for reading
in \tt{SymBandMatrix}es.  Again, this should be included in the object file where 
I use it, \texttt{TMV\_SymBandMatrix.o}.

The problem is probably due to something not being installed correctly on that
computer, or maybe I just did not include the correct linkages or something.  
But rather than trying to get the sysadmin for the computer to find and fix the problem, 
I just added an option to compile with a simple median-of-three
quicksort algorithm, rather than the STL \tt{sort} command, and to read the strings
in character by character.  You can use this option by compiling with the 
flag \texttt{-DNOSTL}.  

\item {\bf Non-portable IsNaN():}
The LAPACK function \tt{sstegr} sometimes produces \tt{nan} values on output.
Apparently there is a bug in at least some distributions of this function.
Anyway, TMV checks for this and calls the alternative (but slower) function
\tt{sstedc} instead whenever a \tt{nan} is found.  
The problem is that there is no C++ standard way to check
for a \tt{nan} value.  

The usual way is to use a macro \tt{isnan}, which is usually
defined in the file \tt{<math.h>}.  However, this is technically a C99 extension,
not standard C++.  So if this macro is not defined, then TMV tries two other
tests that usually detect \tt{nan} correctly.  But if this doesn't work correctly
for you, then you may need to edit the file \texttt{src/TMV\_IsNaN.cpp} to work
with your system\footnote{
However, the provided code did work successfully on all the compilers I 
tested it on, so technically this is not a ``known'' compiler issue, just a 
potential issue.}.

Alternatively, you can compile with \texttt{-DNOSTEGR} which will always
use \tt{sstedc} instead of \tt{sstegr} at the expense of a bit of speed.
Since this is the only place we need to use the \tt{IsNaN} function, that 
should fix the problem.

\end{itemize}

%\begin {comment}

\section{Example Code}

Here are five complete programs that use the TMV library.  They are intended to showcase
some of the features of the library, but they are certainly not comprehensive.  Hopefully,
they will be useful for someone who is just getting started with the TMV library as a 
kind of tutorial of the basics.


The code includes the output as comments that start with \lstinline{//!}, so you can more easily see what is going on.
And each file listed here is also included with the TMV distribution in the \texttt{examples}
directory, so you can easily compile and run them yourself.

\subsection{Vector}

\inputcode{../examples/Vector.cpp}
%\lstinputlisting[basicstyle=\small]{../examples/Vector.cpp}
%\verbatiminput{../examples/Vector.cpp}
\vspace{12pt}

\subsection{Matrix}

\inputcode{../examples/Matrix.cpp}
%\lstinputlisting[basicstyle=\small]{../examples/Matrix.cpp}
%\verbatiminput{../examples/Matrix.cpp}
\vspace{12pt}

\subsection{Division}

\inputcode{../examples/Division.cpp}
%\lstinputlisting[basicstyle=\small]{../examples/Division.cpp}
%\verbatiminput{../examples/Division.cpp}
\vspace{12pt}

\subsection{BandMatrix}

\inputcode{../examples/BandMatrix.cpp}
%\lstinputlisting[basicstyle=\small]{../examples/BandMatrix.cpp}
%\verbatiminput{../examples/BandMatrix.cpp}
\vspace{12pt}

\subsection{SymMatrix}

\inputcode{../examples/SymMatrix.cpp}
%\lstinputlisting[basicstyle=\small]{../examples/SymMatrix.cpp}
%\verbatiminput{../examples/SymMatrix.cpp}

%\end{comment}

\newpage
\section{Known Bugs and Deficiencies (aka To Do List)}

Here is a list of various problems with the current version of the TMV library.
These are mostly features that are not yet included, rather than bugs per se.
If you are looking for issues with getting TMV compiled correctly, you should
probably see \S\ref{compilerissues} instead where I discuss various known
problems that are specific to particular compilers or BLAS or LAPACK 
distributions.  

If you find something to add to this list, or if you want me to bump something
to the top of the list, let me know.  Not that the list is currently in any kind of 
priority order, but, you know what I mean.  Email me at \myemail.

\begin{enumerate}
\item
\textbf{Symmetric arithmetic}
\index{SymMatrix!arithmetic}

When writing complicated equations involving complex symmetric or hermitian matrices, 
you may find that an equation that seems perfectly ok does not compile.
The reason for this problem is explained in \S\ref{symarith} in some detail, 
so you should read about it there.  But basically, the workaround is usually
to break your equation up into smaller steps that do not require the code to 
explicitly instantiate any matrices.  For example: (this is the example from \S\ref{symarith})
\begin{tmvcode}
s3 += x*s1 + s2;
\end{tmvcode}
will not compile if \tt{s1}, \tt{s2}, and \tt{s3} are all complex symmetric, even though it is 
valid, mathematically.  Rewriting this as:
\begin{tmvcode}
s3 += x*s1;
s3 += s2;
\end{tmvcode}
will compile and work correctly.

\item
\textbf{Eigenvalues and eigenvectors}
\index{Eigenvalues}

The code only finds eigenvalues and eigenvectors for hermitian matrices.
I need to add the non-hermitian routines.

\item
\textbf{More special matrix varieties}

Block-diagonal, generic sparse (probably both
row-based and column-based), block sparse, symmetric and hermitian block
diagonal, small symmetric and hermitian...
Maybe skew-symmetric and skew-hermitian.  Are these worth adding?  Let me know.

\item
\textbf{Packed storage}
\index{SymMatrix!packed storage}
\index{UpperTriMatrix!packed storage}

Triangle and symmetric matrices. can be stored in (approximately) half the 
memory as a full $N \times N$ matrix using what is known as packed storage.  
There are BLAS routines for dealing with
these packed storage matrices, but I don't yet have the ability to 
create/use such matrices.

\item
\textbf{List initialization for special matrices}
\index{ListInit}

I'm not sure what makes the most sense for the list initialization of special matrices.
The most straightforward way to implement it would be to just assign the elements
in order that they are stored in memory.  But what do I do for the memory locations
that do not correspond to a location in memory.
For example, for an \tt{UpperTriMatrix}, should the initializer be:
\begin{tmvcode}
U = tmv::ListInit,
    1, 2, 3,
    0, 4, 5,
    0, 0, 6;
\end{tmvcode}
or
\begin{tmvcode}
U = tmv::ListInit,
    1, 2, 3,
       4, 5,
          6;
\end{tmvcode}
Likewise, a \tt{BandMatrix} has confusing locations for elements that are allocated in 
memory, but are not actually part of the matrix.  Should I require these to be in the 
list of values?  If you have any thoughts about this, feel free to email me about them.

\item
\textbf{Hermitian eigenvector algorithm}
\index{SymMatrix!eigenvalues and eigenvectors}
\index{SymBandMatrix!eigenvalues and eigenvectors}

There is a faster algorithm for calculating eigenvectors of a hermitian
matrix given the eigenvalues, which uses a technique know as
a ``Relatively Robust Representation''.  The native TMV
code does not use this, so it is slower than a compilation which calls
the LAPACK routine.
I think this is the only routine for which the LAPACK version is still significantly
faster than the native TMV code.

\item
\textbf{Row-major Bunch-Kaufman}
\index{SymMatrix!Bunch-Kaufman decomposition}
\index{Bunch-Kaufman Decomposition!SymMatrix}

The Bunch-Kaufman decomposition for row-major symmetric/hermitian
matrices is currently $L D L^\dagger$, rather than $L^\dagger D L$.  
The latter should be somewhat (30\%?) faster.  The current $L D L^\dagger$
algorithm is the faster algorithm for column-major matrices.\footnote{
These comments hold when the storage of the symmetric matrix is in the 
lower triangle - it is the opposite for storage in the upper triangle.}

\item
\textbf{Conditions}

Currently, the SVD is the only decomposition that calculates the condition
of a matrix (specifically, the 2-condition).  
LAPACK has routines to calculate the 1- and infinity-condition
from an LU decomposition (and others).  I should add a similar capability.

\item
\textbf{Division error estimates}

LAPACK provides these.  It would be nice to add something along the same lines.

\item
\textbf{Equilibrate matrices}

LAPACK can equilibrate matrices before division.  Again, I should include this
feature too.  Probably as an option (since most matrices don't need it)
as something like \tt{m.Equilibrate()} before calling a division routine.

\item
\textbf{OpenMP}
\index{OpenMP}

I have rewritten the basic Matrix product algorithm to exploit multiple threads using 
OpenMP pragma's.  This gives a good boost in speed for non-BLAS 
compilations, especially since most of the calculation time for the higher-level
algorithms is in the MultMM functions.  Also, the SVD divide and conquer
algorithm uses OpenMP.  

But the SVD only uses half of the potential for OpenMP at the moment.  I need
to reorganize the algorithm a bit to make it more amenable to being parallelized,
but it is certainly doable.

Also I should go through the whole code to
see which other algorithms might benefit from parallelization.  I suspect that all of
the so-called ``Level 3 BLAS'' functions will be amenable to parallelization.  I'm not sure
which higher level functions (i.e. those normally in a LAPACK library) can
be parallelized, but probably some of them.

\item
\textbf{Check for memory throws}

Many algorithms are able to increase their speed by allocating extra
workspace.  Usually this workspace is significantly smaller than the
matrix being worked on, so we assume there is enough space for 
these allocations.  However, I should add try-catch blocks to catch 
any out-of-memory throws and use a less memory-intesive algorithm
when necessary.

\item
\textbf{Overflow and underflow}

For some algorithms, I check for overflow and underflow problems.
(e.g. \tt{Norm(m)}), but I have not put potentially problematic matrices
into the test suite to systematically search for such problems.  So there still
may be places where matrices with very large or small values are a problem.

\item
\textbf{Permutations}

I should make it easier to use the permutations that are output from 
Sort(), as well as from LU and QRP decompositions.   I'd like to write
\tt{m *= P}, rather than \tt{m.PermuteCols(p)}.  This just needs to be a wrapper
class for the functions that already exist, so it shouldn't be too hard to write.

\end{enumerate}

\newpage
\section{History}
\label{history}

Here is a list of the changes from version to version.  Whenever a change 
is not backward compatible, meaning that code using the previous version
might be broken, I mark the item with a $\times$ bullet rather than the usual $\bullet$
to indicate this.  
Also, the bulleted lists are not comprehensive.  
In most cases, new versions fix minor bugs that I find in the old version.  I only 
list the more significant changes.

\begin{description}
\item[Version 0.1]
The first matrix/vector library I wrote.  It wasn't very good, really. 
It had a lot of the functionality I needed, like mixing complex/real, 
SV decomposition, LU decomposition, etc.  But it wasn't at all fast for 
large matrices.  It didn't call BLAS or LAPACK, nor were the native routines
very well optimized.  Also, while it had vector views for rows and columns, 
it didn't have matrix views for things like transpose.  Nor did it have any
delayed arithmetic evaluation.  And there were no special matrices.

I didn't actually name this one 0.1 until I had what I called version 0.3.

\item[Version 0.2]
This was also not named version 0.2 until after the fact.  It had most of 
the current interface for regular Matrix and Vector operations.  I added
Upper/Lower TriMatrix and DiagMatrix.  It also had matrix views and
matrix composites to delay arithmetic evaluation.  The main problem was that
it was still slow.  I hadn't included any BLAS calls yet.  And while the 
internal routines at least used algorithms that used unit strides whenever 
possible, they didn't do any blocking or recursion, which are key for large
matrices.

\item[Version 0.3]
Finally, I actually named this one 0.3 at the time.
The big addition here was BLAS and LAPACK calls, which 
helped me to realize how slow my internal code really was (although I hadn't
updated them to block or recursive algorithms yet).
I also added BandMatrix.

\item[Version 0.4]
The new version number here was because I needed some added functionality for a
project I was working on.  It retrospect, it really only deserves a 0.01 increment,
since the changes weren't that big.  But, oh well.
\begin{itemize}
\item
Added QR\_Downdate.  (This was the main new functionality I needed.)
\item
Improved the numerical accuracy of the QRP decomposition.
\item
Added the possibility of not storing U,V for the SVD.
\item
Greatly improved the test suite, and consequently found and corrected a few bugs.
\item
Added some arithmetic functionality that had been missing (like \tt{m += L*U}).
\end{itemize}

\item[Version 0.5]
The new symmetric matrix classes precipitated a major version number update.
I also sped up a lot of the algorithms:
\begin{itemize}
\item
Added SymMatrix, HermMatrix, and all associated functionality.
\item
Added blocked versions of most of the algorithms, so the non-LAPACK
code runs a lot faster.
\item
Allowed for loose QRP decomposition.
\item
Added DivideInPlace().
\end{itemize}

\item[Version 0.51]
Some minor improvements:
\begin{itemize}
\item
Sped up some functions like matrix addition and assignment by adding the 
LinearView method.
\item
Added QR\_Update, and improved the QR\_Downdate algorithm.
\item
Blocked some more algorithms like TriMatrix multiplication/division, so non-BLAS
code runs significantly faster (but still slower than BLAS).
\end{itemize}

\item[Version 0.52]
The first ``public'' release!  And correspondingly, the first with documentation
and a web site.  A few other people had used previous versions, but since the
only documentation was my comments in the .h files, it wasn't all that user-friendly.
\begin{itemize}
\item
Added SaveDiv() and related methods for division control.  Also changed the 
default behavior from saving the decomposition to not saving it.
\item
Added in-place versions of the algorithms for $S = L^\dagger L$ and $S = L L^\dagger$.
\end{itemize}

\item[Version 0.53]
By popular demand (well, a request by Fritz Stabenau, at least):
\begin{itemize}
\item
Added the Fortran-style indexing.
\end{itemize}

\item[Version 0.54]
Inspired by my to-do list, which I wrote for Version 0.52, I tackled a few of the 
items on the list and addressed some issues that people had been having
with compiling:
\begin{itemize}
\item[$\times$]
Changed from a rudimentary exception scheme (with just one class - 
\tt{tmv\_exception}) to the current more full-featured exception hierarchy.
Also added \tt{auto\_ptr} usage instead of bald pointers to make the 
exception throws memory-leak safe.
\item
Sped up SymLUDiv and SymSVDiv inverses.
\item
Added the possibility of compiling a small version of the library and test suite.
\item[$\times$]
Consolidated SymLUDiv and HermLUDiv classes into just SymLUDiv, which now checks
whether the matrix is hermitian automatically.  
\item
Reduced the number of operations that make temporary matrices when multiple
objects use the same storage. 
\item
Specialized Tridiagonal $\times$ Matrix calculation.
\item
Added ElementProd and AddElementProd functions for matrices.
\item
Added CLAPACK and ACML as known versions of LAPACK (the LAPACK calls
had previously been specific to the Intel MKL version).  Also made the file
\tt{TMV\_Blas.h} much better organized, so it is easier to tailor the code to 
a different LAPACK distribution with different calling conventions.

\end{itemize}

\item[Version 0.60]
This revision merits a first-decimal-place increment, since I added a 
few big features.  I also registered it with SourceForge, 
which is a pretty big milestone as well.
\begin{itemize}
\item
Added \tt{SmallVector} and \tt{SmallMatrix} with all accompanying algorithms.
\item 
Added \tt{SymBandMatrix} and \tt{HermBandMatrix} with all accompanying algorithms.
\item
Made arithmetic between any two special matrices compatible, so long as 
the operation is allowed given their respective shapes.  
e.g. \tt{U += B} and \tt{U *= B} are allowed if \tt{B} is upper-banded.  
There are many other expressions that are now legal statements.
\item[$\times$]
Changed \tt{QR\_Downdate()} to throw an exception rather than return false
when it fails.
\item
Added the GPL License to the end of this documentation, and a copyright
and GPL snippet into each file.
\item[$\times$]
Changed the -D compiler options for changing which types get instantiated.
The default is still to instantiate double and float.  Now to turn these
off, use \texttt{-DNO\_INST\_DOUBLE} and \texttt{-DNO\_INST\_FLOAT} respectively.  To
add int or long double instantiations, use \texttt{-DINST\_INT} and 
\texttt{-DINST\_LONGDOUBLE} respectively.
\item
Split up the library into \texttt{libtmv.a} and \texttt{libtmv\_symband.a}.  The
latter includes everything for the \tt{SymMatrix}, \tt{HermMatrix}, 
\tt{BandMatrix}, \tt{SymBandMatrix}, and \tt{HermBandMatrix} classes.
The former includes everything else including \tt{DiagMatrix} and
\tt{Upper/LowerTriMatrix}.  Also, I got rid of \texttt{libsmalltmv.a}, which
was basically the same as the new trimmed down \texttt{libtmv.a}.

\end{itemize}

\item[Version 0.61]
A number of updates mostly precipitated by feature requests by me in my own
use of the library, as well as some from a few other users.
I also did a complete systematic edit of the documentation which 
precipitated some more changes to make the UI a bit more intuitive.
\begin{itemize}
\item[$\times$]
Changed the default storage for the \tt{Matrix} class to \tt{ColMajor}
rather than \tt{RowMajor}, since it seems to be more common that column-major
storage is the more efficient choice.  Therefore, it makes sense to have this
be the default.
\item[$\times$]
Changed a lot of \tt{size\_t} parameters to \tt{int} - mostly variables
which are indices of a matrix or vector, like \tt{i} and \tt{j} in \tt{m(i,j)}.  
These variables used to be of type \tt{size\_t}, and now they are \tt{int},
even though they are usually required to be positive.

The reason for this change
was that the implementation of these
functions often involves multiplying the index
by a step size, which is allowed to be negative (and hence
is an \tt{int}), so there were lots of casts to \tt{int} for these variables.
I decided that it would be better to simply have them be \tt{int} in the first place.

The particular change that is most likely to require modification to existing code
involves permutations, which used to be \tt{size\_t} arrays, and are now
\tt{int} arrays.  So if you used them, you might need to change their
declarations in your code to \tt{(int [])}.
\item[$\times$]
Removed \tt{U.MakeUnitDiag}, since the name was counter-intuitive to what itactually did, and the functionality is more obvious through the
\tt{UpperTriMatrixViewOf} function.
\item
Sped up matrix multiplication for non-blas implementations, including
openmp pragmas to allow for multiple threads on machines that support them.
The code is now within a factor of 2 or 3 of a good optimized BLAS 
distribution.  So it is still worth it to use BLAS if you have one available,
but if you don't have one on you machine, the non-blas code is no longer a
order of magnitude slower.
\item
Changed a few things which prevented Microsoft Visual C++ from compiling successfully.  
As far as I can tell, these are aspects in which VC++ is not fully compliant with the C++ 
standard, but there were work-arounds for all of them.
Thanks to Andy Molloy
for spearheading this effort and doing the lion's share of the work to make
the code compatible with the VC++ compiler.
\item[$\times$]
Removed the optional index parameter to the non-method versions of 
MaxElement, etc.   i.e.\\
\tt{vmax = v.MaxElement(\&imax)}\\
will work, but not\\
\tt{vmax = MaxElement(v,\&imax)}\\
However the functional form without the \tt{imax} parameter still works as before.
Basically, this was just a semantic choice.  It seems to me that the meaning of the method
form with the index parameter is much clearer than the functional form with two arguments.

I also added an optional \tt{scale} parameter to \tt{m.NormSq(scale)}.
Again, this optional parameter is only allowed in the method version, not the function \tt{NormSq(m)}
\item[$\times$]
Added the explicit decomposition routines.  I also
got rid of the \tt{SVU}, \tt{SVV} and \tt{SVS} options for \tt{m.DivideUsing(...)}, since the point 
of these was to do the decomposition without calculation $U$ and/or $V$.  This is now done
more intuitively with the explicit decomposition routines.
I also added the (hermitian) eigenvalue/eigenvector routines which used to
require using the division accessors in non-intuitive ways to calculate.
\item 
Fixed a couple of places where underflow and overflow could cause problems.  However, I have
not put such very large or very small matrices into the test suite, so there are probably other
places where these could be a problem.  (Added this to the To Do list.)
\item
Updated the native TMV code for the singular value decomposition and hermitian 
eigenvalue calculation to use the divide-and-conquer algorithm.
\item
Added \tt{m.LogDet()} method.  With very large matrices, it is common for the determinant
to overflow, but often the logarithm of the determinant is sufficient.  
For example, one may be trying to minimize a 
likelihood function that
includes the determinant of a matrix.  Minimizing the log(likelihood) is equivalent, which
thus involves the log of the determinant.  
\item[$\times$]
Changed \tt{m.SVD().GetS()} to return a \tt{DiagMatrix} rather than a \tt{Vector}.
(Sorry - this change eluded documentation until version 0.62.)
\end{itemize}

\item[Version 0.62]

This release contains a number of bug fixes and corrections of mistakes in the documentation.
I also significantly revamped the \tt{SmallMatrix} class to make it faster at the expense of a few fewer 
features.  
\begin{itemize}
\item
Corrected an error with \tt{m.DivIsSet()}.
\item
Corrected some errors in the documentation -- Thanks to Jake VanderPlas for pointing these out.
\item
Improved the behavior of the division accessors, also at the suggestion of Jake VanderPlas. (Thanks again!)  They used to require that the division
object had already been set or they would give an error.  So to set the SVD threshold for example,
you had to write:\\
\tt{m.DivideUsing(tmv::SV);}\\
\tt{m.SetDiv();}\\
\tt{m.SVD().Thresh(1.e-5);}\\
Jake rightly pointed out that this was not very intuitive.  So now the accessors (like \tt{m.SVD()}) set
the division object appropriately if it is not already set.  The above task is now simply performed
by just the single line:\\
\tt{m.SVD().Thresh(1.e-5);}
\item
Added the \tt{ListInit} method for initializing the values of a \tt{Vector} or \tt{Matrix}.  Thanks to Paul
Sarli for this suggestion.  The implementation is based somewhat on a similar 
initialization syntax in the Little Template Library (LTL) by Niv Drory et al.
\item[$\times$]
Significantly changed the \tt{SmallMatrix} class.  See \S\ref{smallmatrix}, on the \tt{SmallMatrix} class 
in the documentation
for the full details of how it works now.  But in brief, here are the main changes:
\begin{itemize}
\item \tt{SmallMatrix} and \tt{SmallVector} no longer inherit from \tt{GenMatrix} and \tt{GenVector}.
\item This allowed us to remove all virtual functions and the corresponding \tt{vtable}.
\item Improved the arithmetic so more of the routines correctly do the calculation inline to allow the
compiler more opportunity to optimize the calculation.
\item Added inlined division and determinants.
\item Got rid of the "Small" views.  Because I got rid of the inheritance in \tt{SmallMatrix}, there is no longer a 
\tt{GenSmallMatrix} class from which the views and regular objects can both inherit.  This
makes it a bit unwieldy to write all the different arithmetic combinations of how 
\tt{SmallMatrix} and \tt{SmallMatrixView} could combine.  So I just made all views
regular \tt{VectorView} or \tt{MatrixView} objects.  These can be assigned to a \tt{SmallVector}
or \tt{SmallMatrix}, or used in arithmetic calculations with them, so all the arithmetic statements
that use \tt{Transpose()} or \tt{row()} or \tt{col()} (for example) will still work, but they might
not be done inline anymore.
\end{itemize}
\item
Consolidated some of the old header files.  Now all \tt{TriMatrix} and \tt{DiagMatrix}
calculations are included as part of \tt{TMV.h}, so there is no need to include \tt{TMV\_Tri.h}
or \tt{TMV\_Diag.h} anymore.  Although, these header files are still included for backwards
compatibility.
\item[$\times$]
Removed some of the \tt{ViewOf} commands that were not very clear and
which have other, clearer ways of doing the same thing:
\begin{itemize}
\item
\tt{d = DiagMatrixViewOf(m)} should now be written\\
\tt{d = DiagMatrixViewOf(m.diag())}.
\item
\tt{U = UpperTriMatrixViewOf(m)} should now be written\\
\tt{U = m.UpperTri()}.
\item
\tt{U = UpperTriMatrixViewOf(m,UnitDiag)} should now be written\\
\tt{U = m.UpperTri(UnitDiag)}.
\item
\tt{U = UpperTriMatrixViewOf(U,UnitDiag)} should now be written\\
\tt{U = U.ViewAsUnitDiag()}.
\end{itemize}
\item
Tracked down the problems I had been having with the LAPACK \tt{dstegr} and \tt{sstegr} 
functions.  The segmentation faults were because the algorithm writes to an element
one pas the end of the E vector.  Apparently, this is a documented feature, but it doesn't
make any sense to me why they would need to do this.  I had previous discovered similar
behavior with the \tt{zhetrd} and \tt{dgebrd} functions.  Anyway, I fixed this by allocating a
vector with one extra element and copying the values.

I also found that there are two separate problems with the \tt{stegr} algorithm where
it produces incorrect results sometimes.
First, sometimes the \tt{stegr} algorithm seems to return early - before the last update
to the D vector and U matrix.  I actually figured out how to correct the D vector from some 
values that remain in the workspace, but I couldn't figure out how to correct the U matrix.
This is also a documented feature -- \tt{stegr} returns the value of 2 in the \tt{info} variable.

Second, the \tt{stegr} algorithm is apparently not very careful about avoiding overflow or underflow.
I found \tt{NaN}'s in the output U matrix for particular input matrices from the float version
of the algorithm, \tt{sstegr}.  The \tt{info} variable is 0 on output in these cases, so unfortunately
\tt{stegr} does not detect the problem.  So this has to be checked for by hand.

Since both of these problems are detectable based on the output values (including examining
the workspace in the first case), TMV now checks for the problem, and it there was a problem,
it calls the \tt{stedc} routine instead.
\item
Tested the code for memory bugs with Paul Nettle's mmgr.h code.  There were only a couple of 
minor memory leaks, which were fixed.
\item
Fixed a problem with the OpenMP version of the code giving segmentation faults with pgCC.
\item
Added the SCons installation method to automatically discover what BLAS and LAPACK libraries 
are installed on your computer.  This should make it easier for most users to install an efficient
version of the TMV library without having to fiddle with the Makefile.
\item
Added compatibility for Fortran versions of BLAS and LAPACK, including header files that
should work with them, \texttt{fblas.h} and \texttt{flapack.h} are in the \texttt{include} directory.
\item
Added the CMake installation method.  Thanks to Andy Molloy for providing this.  Unfortunately,
it is not as full featured as the SCons method, but it is pretty straightforward to get
a basic TMV library compiled.
\item
Added hyperlinks to the PDF documentation.  This should make it much easier to locate a particular topic by simply clicking on the link in the Table of Contents.  In PDF viewers that support it, you can 
also navigate using the "drawer" which will show the various sections and let you jump directly to them.

Also, I added an index, complete with hyperlinks as well, that should make it easier to find a particular topic that you might need help with.
\item
Fixed other miscellaneous bugs.

\end{itemize}


\end{description}

\newpage
\section{License}
\label{license}
\index{License}
\index{GNU General Public License}

This software is licensed under the GNU General Public License.  This license is
sometimes referred to as ``copyleft''.  This gist of the license (made more specific
in the following text) is that you are allowed to copy, modify, and use TMV code
with basically no restrictions as far as your own use is concerned, 
but with a few restrictions regarding further distribution.

The biggest restriction is that if you distribute code that
uses the TMV code, then you must distribute it with a copyleft license as well.
In other words, you may not use the TMV library with commercial, proprietary 
software. 

The other main restriction on such a distribution is that you must include this 
document,
including the below license, along with your version of the code.  If you modified
any portions of the library, you must indicate your modifications in the accompanying
documentation files.

More information may be obtained from \url{http://www.gnu.org/copyleft/}.

\begin{center}
{\parindent 0in

Version 2, June 1991

Copyright \copyright\ 1989, 1991 Free Software Foundation, Inc.

\bigskip

51 Franklin St, Fifth Floor, Boston, MA  02110-1301, USA

\bigskip

Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
}
\end{center}

\renewcommand{\abstractname}{Preamble}
\begin{abstract}
The licenses for most software are designed to take away your freedom to
share and change it.  By contrast, the GNU General Public License is
intended to guarantee your freedom to share and change free software---to
make sure the software is free for all its users.  This General Public
License applies to most of the Free Software Foundation's software and to
any other program whose authors commit to using it.  (Some other Free
Software Foundation software is covered by the GNU Library General Public
License instead.)  You can apply it to your programs, too.

When we speak of free software, we are referring to freedom, not price.
Our General Public Licenses are designed to make sure that you have the
freedom to distribute copies of free software (and charge for this service
if you wish), that you receive source code or can get it if you want it,
that you can change the software or use pieces of it in new free programs;
and that you know you can do these things.

To protect your rights, we need to make restrictions that forbid anyone to
deny you these rights or to ask you to surrender the rights.  These
restrictions translate to certain responsibilities for you if you
distribute copies of the software, or if you modify it.

For example, if you distribute copies of such a program, whether gratis or
for a fee, you must give the recipients all the rights that you have.  You
must make sure that they, too, receive or can get the source code.  And
you must show them these terms so they know their rights.

We protect your rights with two steps: (1) copyright the software, and (2)
offer you this license which gives you legal permission to copy,
distribute and/or modify the software.

Also, for each author's protection and ours, we want to make certain that
everyone understands that there is no warranty for this free software.  If
the software is modified by someone else and passed on, we want its
recipients to know that what they have is not the original, so that any
problems introduced by others will not reflect on the original authors'
reputations.

Finally, any free program is threatened constantly by software patents.
We wish to avoid the danger that redistributors of a free program will
individually obtain patent licenses, in effect making the program
proprietary.  To prevent this, we have made it clear that any patent must
be licensed for everyone's free use or not licensed at all.

The precise terms and conditions for copying, distribution and
modification follow.
\end{abstract}

\begin{center}
{\Large \sc GNU General Public License
\\\vspace{3mm}Terms and Conditions For Copying, Distribution and Modification}
\end{center}


\begin{enumerate}

\addtocounter{enumi}{-1}

\item 

This License applies to any program or other work which contains a notice
placed by the copyright holder saying it may be distributed under the
terms of this General Public License.  The ``Program'', below, refers to
any such program or work, and a ``work based on the Program'' means either
the Program or any derivative work under copyright law: that is to say, a
work containing the Program or a portion of it, either verbatim or with
modifications and/or translated into another language.  (Hereinafter,
translation is included without limitation in the term ``modification''.)
Each licensee is addressed as ``you''.

Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope.  The act of
running the Program is not restricted, and the output from the Program
is covered only if its contents constitute a work based on the
Program (independent of having been made by running the Program).
Whether that is true depends on what the Program does.

\item You may copy and distribute verbatim copies of the Program's source
  code as you receive it, in any medium, provided that you conspicuously
  and appropriately publish on each copy an appropriate copyright notice
  and disclaimer of warranty; keep intact all the notices that refer to
  this License and to the absence of any warranty; and give any other
  recipients of the Program a copy of this License along with the Program.

You may charge a fee for the physical act of transferring a copy, and you
may at your option offer warranty protection in exchange for a fee.

\item

You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:

\begin{enumerate}

\item 

You must cause the modified files to carry prominent notices stating that
you changed the files and the date of any change.

\item

You must cause any work that you distribute or publish, that in
whole or in part contains or is derived from the Program or any
part thereof, to be licensed as a whole at no charge to all third
parties under the terms of this License.

\item
If the modified program normally reads commands interactively
when run, you must cause it, when started running for such
interactive use in the most ordinary way, to print or display an
announcement including an appropriate copyright notice and a
notice that there is no warranty (or else, saying that you provide
a warranty) and that users may redistribute the program under
these conditions, and telling the user how to view a copy of this
License.  (Exception: if the Program itself is interactive but
does not normally print such an announcement, your work based on
the Program is not required to print an announcement.)

\end{enumerate}


These requirements apply to the modified work as a whole.  If
identifiable sections of that work are not derived from the Program,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works.  But when you
distribute the same sections as part of a whole which is a work based
on the Program, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote it.

Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Program.

In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.

\item
You may copy and distribute the Program (or a work based on it,
under Section 2) in object code or executable form under the terms of
Sections 1 and 2 above provided that you also do one of the following:

\begin{enumerate}

\item

Accompany it with the complete corresponding machine-readable
source code, which must be distributed under the terms of Sections
1 and 2 above on a medium customarily used for software interchange; or,

\item

Accompany it with a written offer, valid for at least three
years, to give any third party, for a charge no more than your
cost of physically performing source distribution, a complete
machine-readable copy of the corresponding source code, to be
distributed under the terms of Sections 1 and 2 above on a medium
customarily used for software interchange; or,

\item

Accompany it with the information you received as to the offer
to distribute corresponding source code.  (This alternative is
allowed only for noncommercial distribution and only if you
received the program in object code or executable form with such
an offer, in accord with Subsection b above.)

\end{enumerate}


The source code for a work means the preferred form of the work for
making modifications to it.  For an executable work, complete source
code means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to
control compilation and installation of the executable.  However, as a
special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary
form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component
itself accompanies the executable.

If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent
access to copy the source code from the same place counts as
distribution of the source code, even though third parties are not
compelled to copy the source along with the object code.

\item
You may not copy, modify, sublicense, or distribute the Program
except as expressly provided under this License.  Any attempt
otherwise to copy, modify, sublicense or distribute the Program is
void, and will automatically terminate your rights under this License.
However, parties who have received copies, or rights, from you under
this License will not have their licenses terminated so long as such
parties remain in full compliance.

\item
You are not required to accept this License, since you have not
signed it.  However, nothing else grants you permission to modify or
distribute the Program or its derivative works.  These actions are
prohibited by law if you do not accept this License.  Therefore, by
modifying or distributing the Program (or any work based on the
Program), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Program or works based on it.

\item
Each time you redistribute the Program (or any work based on the
Program), the recipient automatically receives a license from the
original licensor to copy, distribute or modify the Program subject to
these terms and conditions.  You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties to
this License.

\item
If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License.  If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Program at all.  For example, if a patent
license would not permit royalty-free redistribution of the Program by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Program.

If any portion of this section is held invalid or unenforceable under
any particular circumstance, the balance of the section is intended to
apply and the section as a whole is intended to apply in other
circumstances.

It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system, which is
implemented by public license practices.  Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.

This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.

\item
If the distribution and/or use of the Program is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Program under this License
may add an explicit geographical distribution limitation excluding
those countries, so that distribution is permitted only in or among
countries not thus excluded.  In such case, this License incorporates
the limitation as if written in the body of this License.

\item
The Free Software Foundation may publish revised and/or new versions
of the General Public License from time to time.  Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.

Each version is given a distinguishing version number.  If the Program
specifies a version number of this License which applies to it and ``any
later version'', you have the option of following the terms and conditions
either of that version or of any later version published by the Free
Software Foundation.  If the Program does not specify a version number of
this License, you may choose any version ever published by the Free Software
Foundation.

\item
If you wish to incorporate parts of the Program into other free
programs whose distribution conditions are different, write to the author
to ask for permission.  For software which is copyrighted by the Free
Software Foundation, write to the Free Software Foundation; we sometimes
make exceptions for this.  Our decision will be guided by the two goals
of preserving the free status of all derivatives of our free software and
of promoting the sharing and reuse of software generally.

\begin{center}
{\Large\sc
No Warranty
}
\end{center}

\item
{\sc Because the program is licensed free of charge, there is no warranty
for the program, to the extent permitted by applicable law.  Except when
otherwise stated in writing the copyright holders and/or other parties
provide the program ``as is'' without warranty of any kind, either expressed
or implied, including, but not limited to, the implied warranties of
merchantability and fitness for a particular purpose.  The entire risk as
to the quality and performance of the program is with you.  Should the
program prove defective, you assume the cost of all necessary servicing,
repair or correction.}

\item
{\sc In no event unless required by applicable law or agreed to in writing
will any copyright holder, or any other party who may modify and/or
redistribute the program as permitted above, be liable to you for damages,
including any general, special, incidental or consequential damages arising
out of the use or inability to use the program (including but not limited
to loss of data or data being rendered inaccurate or losses sustained by
you or third parties or a failure of the program to operate with any other
programs), even if such holder or other party has been advised of the
possibility of such damages.}

\end{enumerate}


\begin{center}
{\Large\sc End of Terms and Conditions}
\end{center}

This library is largely based upon work supported by the 
National Aeronautics and Space Administration (NASA) under Grant No 
NAG5-10924, a Keck Foundation grant, and the University of Pennsylvania. 
Any opinions, findings and conclusions or recommendations expressed in this 
material are those of the author(s) and do not necessarily reflect the views 
of NASA, the Keck Foundation, or the University of Pennsylvania.
This software is made available to you on an ``as is'' basis with no 
representations or warranties, express or implied, including but not 
limited to any warranty of performance, merchantability, fitness for a 
particular purpose, commercial utility, non-infringement or title.  
Neither the author nor the organizations providing the support under 
which the work was developed will be liable to you or any third party 
with respect to any claim arising from your further development of the 
software or any products related to or derived from the software, or for 
lost profits, business interruption, or indirect special or consequential 
damages of any kind. 

\newpage
\printindex

\end{document}


